[  462.612849] env[60890]: INFO neutron.common.config [-] Logging enabled!
[  462.613345] env[60890]: INFO neutron.common.config [-] /opt/stack/data/venv/bin/neutron-nsxv3-agent version 24.0.0.0rc2.dev59
[  462.613345] env[60890]: DEBUG neutron.common.config [-] command line: /opt/stack/data/venv/bin/neutron-nsxv3-agent --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini --config-file /etc/neutron/plugins/ml2/ml2-nsxv3-1.ini {{(pid=60890) setup_logging /opt/stack/neutron/neutron/common/config.py:123}}
[  462.613563] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] VMware NSXv3 Agent initializing ...
[  462.613658] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [-] Eventlet blocking behavior detection initialization completed.
[  462.614492] env[60890]: WARNING oslo_policy.policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  462.614738] env[60890]: WARNING oslo_policy.policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] JSON formatted policy_file support is deprecated since Victoria release. You need to use YAML format which will be default in future. You can use ``oslopolicy-convert-json-to-yaml`` tool to convert existing JSON-formatted policy file to YAML-formatted in backward compatible way: https://docs.openstack.org/oslo.policy/latest/cli/oslopolicy-convert-json-to-yaml.html.
[  462.615960] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  462.631252] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Session token - acquiring
[  462.632052] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock https://nsxt.osci.c.eu-de-1.cloud.sap:443 taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 145 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 443 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 207 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_mgmt.py, line 454 in _load_zones>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 276 in get_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 107 in decorator>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/client_nsx.py, line 202 in _login>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  462.632265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  462.632394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  462.632869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  462.642356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "https://nsxt.osci.c.eu-de-1.cloud.sap:443" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  462.902033] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.client_nsx [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Session token - acquired, connected to NSX-T 4.1.2.0.0.22589037
[  462.910796] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Activating Management API Provider.
[  462.925293] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_mgmt [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Looking for the default Switching Profiles.
[  462.925497] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Looking for TransportZone with name nsx-vlan-transportzone.
[  462.942789] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Activating Policy API Provider.
[  462.943002] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Looking for the default Infrastructure Rules.
[  462.996770] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Looking for the Default Layer3 Logged Drop Section.
[  463.009320] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Detected NSX-T (4, 1, 2, 0, 0, 22589037) version.
[  463.009774] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 97 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  463.009967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  463.010098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  463.010397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  463.010544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  463.010780] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock all taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 100 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  463.010942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  463.011107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  463.011371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  463.011631] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_ports_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  463.012490] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Creating reply queue: reply_156db6c4256a4c58884afce15e6f5340
[  463.021481] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Expecting reply to msg 23996d1cbb12412c8d71e7de82698d7f in queue reply_156db6c4256a4c58884afce15e6f5340
[  463.035143] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 23996d1cbb12412c8d71e7de82698d7f
[  463.528967] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_groups_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  463.529572] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Expecting reply to msg 741a6183dc324037851670eb6ed6cffe in queue reply_156db6c4256a4c58884afce15e6f5340
[  463.538582] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 741a6183dc324037851670eb6ed6cffe
[  464.032133] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_qos_policies_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  464.033137] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Expecting reply to msg db0c9207bf734edf81d957851b49c5c7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  464.044997] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg db0c9207bf734edf81d957851b49c5c7
[  464.535618] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Inventory metadata is going to be refreshed.
[  464.536241] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] Fetching Policy NSX-T metadata for Type:Segment.
[  465.223858] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 118 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  465.224826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  465.225212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  465.225671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  465.239317] env[60890]: WARNING networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Duplicate resource with OS_ID: 992 ID: cl2-zone-992
[  465.240353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  465.241416] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] Fetching Policy NSX-T metadata for Type:SegmentPort.
[  465.276622] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock SegmentPort taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 120 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  465.277196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  465.277481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  465.277925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  465.289974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  465.290959] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] The number of outdated resources for Type:SegmentPort Is:96.
[  465.291311] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Outdated resources of Type:SegmentPort Are:{'519bac90-d40c-494c-8eb9-6c53e239f10a', '32332759-ca87-4ef6-955a-325ad700200d', '03fdd0ff-044d-4309-905b-d51a3bab1b81', '55f117f0-ba76-491f-b5c5-7441d3ff7787', '9435dc4a-ead0-4afd-a099-668739ae29f4', '66d98031-6842-491c-9961-c1b4d0fd5d53', '0f875a82-d52b-4617-86ef-09c6a8aeccbb', '2126120d-06ba-46a5-a74c-77f3b18345ab', '17b157f7-bf15-4a5c-a271-9af72764322d', '474946e7-99a3-4290-959e-f2dc1645b999', '4fb6c40c-bf99-4b55-9f6b-ca451ded5287', '7e152039-3f45-45d1-b2a1-c7acb717f68b', '1ca72486-d800-43c5-907b-725bb958b3a3', '2a1bc88a-7641-4017-874c-f15ee8ec3621', 'd357f4a7-7fe1-4677-a32d-b2a6c5780592', '99c860bb-14da-4e95-b94e-d77d15224b7a', 'fb2c7956-257f-4d1d-8dbb-83b5350798c3', 'c4585efa-40da-4b2f-81a8-523efec14826', '7e3eabc0-1383-412e-a875-c4e73daf6808', 'f599de71-cb96-446c-b853-bfcdc671260e', '31125a4c-2800-482d-8420-53a798a29c02', '0cb4488c-6859-4b6c-b322-dc290dbf0f8f', '3e63d897-dbf9-4401-a6d7-1650e7c0ecdf', '62510d77-c23b-4044-a0f1-0f38a92caf2c', '11ab945e-c9e0-4540-8ddf-a57fc4983384', '0ecbd240-2457-42ca-b168-e67fa16f56b3', '156f2622-5bca-4370-8d7a-06130b978cf7', '6f69240a-5122-4e7e-87af-f288a47b5ea9', 'e758e5fd-8995-41a7-9fe7-7ce7894fb7aa', '1e891f4b-00a4-481c-a00c-0320f9e75e62', 'd43c8245-514d-4c94-80b7-c12dfe9a6434', '088ac320-7bf8-4ebf-9318-5c12a3de2e62', '78c543de-6fc1-464e-b0a5-b0db23f19be3', 'ea8f9597-af66-4849-94da-7357f4969707', '889eac8e-299a-4c2b-a8e4-819f8f54a261', '08b26599-acd3-4729-9ae4-7cef241ebca7', '046ad10e-2286-420f-b6f3-0d598b3fcc03', 'bc6fc679-542b-4d2b-838e-70db009ad31a', '64487042-dc1f-4bbb-8b1c-0ff06c270b8d', '45968fc0-6e74-43f7-862b-4043c3249370', '39825d23-3b58-4cdb-8fa7-5a9ff493c789', '2eea5bcc-a0a3-4963-a879-5bf0c1c38f60', '2763deba-ab9e-48b4-b5a8-f7082a993850', 'e9ad7926-1d42-4ff9-aab3-d520de5ae40e', '33359e95-dbc3-4199-9367-e6bd0129468e', '32f3a7a3-308f-4fc7-8898-ed5ad6161587', '88452c84-34f9-49bc-aa27-a6d503e47fa4', 'e1979b45-d8d8-4076-a153-fa64ba78b7c0', 'ace6f305-6029-421e-89d3-bcca9fd2264a', '06f70962-82ab-4995-b7ca-a341783a2c6b', 'ac9ba9d7-c9f6-482f-9cbe-4ac9659c3194', '2da5eafe-fcd1-44dd-b2aa-c1bc16ff3d23', '34dd8f0f-dd08-44fa-9865-37de198ce337', '02696c3f-3dd0-499c-9baa-8a8cf21253eb', 'a385dc0d-17c5-49e9-9685-8285fc885a67', '1b4cdb06-1445-4694-a8a7-25090c7c8982', '74ea2df8-d554-4541-bfb6-8a8fad4756ea', '28285b72-9ad5-4703-950f-d20766e1592d', 'c4c889cc-a303-49e8-b5da-c71c873abb02', '885d71ab-3cf8-44a1-98fe-fedf69f9f021', 'e798c106-4155-4e94-90dc-5591d8705c9e', 'f5808565-bb46-4620-9c27-c24fe7a3d1b6', 'a161c3dd-1d1b-49c2-816f-07c0cc3fe929', '6865e1e1-9000-4b73-b41e-4a4753755582', '271514e3-602e-4224-bfb9-99943b868420', '9674a2da-6ae2-430a-80a2-67bab8177b7e', '8e1ea150-2eec-4dac-80e6-7d1daeaf351d', 'dcde1b76-b76a-4b6b-bbc8-099834069781', '925b62f2-60ee-49b1-974b-66ad3d724498', 'e9d5bf46-3b2c-4088-a6bf-07b65f970ae4', 'f467c1f7-4700-4387-bb2a-249f697bbb6c', '7fcb3b5c-ae38-468f-aecc-8e3293f03e66', 'c4d845cb-64ec-4e2f-87a3-29dcdf4f4b2b', 'c8e81773-32ca-437e-9971-50cef2a264ea', 'd73acfa0-7f7a-4f3e-98ae-e9e2b940b462', 'a367d71c-4063-4a70-9c6c-4f60da50879e', '3903c4b9-6919-4b40-9189-942790370190', 'f1f40c6a-f2c1-4a74-94e2-cdf2e32e24cd', '6040c7d6-38fb-48bc-9161-2b810f0ffb1a', 'fc8312bb-e44e-4d35-b825-f0e74adf2260', 'fa51a02a-4b3e-479b-b703-759a3babfeb1', 'f1446fa4-56d0-44ca-8222-7ed499af9ee1', 'c9f3dd72-2950-4d13-9d1f-d1d7538d8150', 'e3e3d0bf-2787-48ab-9720-f7cb2a5d4ce2', 'f00d4feb-c74c-4e36-ab50-7989164b3eba', 'c5200c6e-cf74-4b1f-8377-44fbfd9e43ac', '48e0c1c5-8a6b-4724-a313-2b27b5d0073d', '194b348c-e895-4f8d-ab94-c33cb6abd8b7', 'c6537245-d078-41cd-b50c-1df8377c8fd6', '0cec1ac8-30f8-403a-a9b6-97dd88f439b7', 'ca735a43-ae10-4d73-851b-23c32b6e90a2', '0e5e87d8-236e-443c-b231-531694a623f3', '10d09f8d-b8e6-456d-85e8-1ecfa1dd206f', 'ed65677d-a1b5-4cb2-a8c2-fd130551ab54', '85bd5f38-c811-4e68-8e48-2bed178d2a9a', 'e937c7a1-6075-47df-be05-bce50cae3909'} {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  465.291743] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Rules).
[  465.945791] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  465.946380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  465.946681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  465.947104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  465.966412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  465.967547] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Rules Remote IP Prefix).
[  468.009949] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  468.010453] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  468.731368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock Security Group (Rules Remote IP Prefix) taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 970 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  468.731571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  468.731659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  468.732274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  468.739830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  468.743895] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] Fetching Policy NSX-T metadata for Type:Address Group.
[  471.524208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock Address Group taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 971 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  471.524732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  471.524732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  471.524931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  471.532680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  471.537244] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] The number of outdated resources for Type:Security Group (Rules) Is:1104.
[  471.537661] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Outdated resources of Type:Security Group (Rules) Are:{'6b38d532-e264-4d27-841f-0e430bb2d26a', '5a93c6de-4566-400e-b225-6fbc784def02', 'aa046b3f-6d44-4acb-8e58-50499e3f8d60', '839be8eb-d3ab-42d2-beae-bb4a6136fea9', '1efe66d8-51ee-4c77-bbc1-b0bff97bd6a0', '83621333-7805-450a-bf42-807dc0903fc0', '02fa8098-695f-4e4c-af31-9d2f20a9fc7c', '2c8665b5-7a16-454b-b09f-403fece4484c', '3c29ea98-dc38-490b-a8e3-abfdc118aa44', 'eb4ebc6c-f4d5-4c08-af40-def126bc447e', '38ad9e85-cb39-4189-9d61-20af96e8fe10', 'a84b8cf6-5370-462a-97e0-5c81e1e22633', 'aeb219c3-95b4-4de5-8fce-a41f8b49395d', '126abd34-fe92-4587-b136-35d78d6b2a21', '80e5b5da-5c33-415d-8683-524bd7b4691a', '2fdf09f9-efa8-4a14-a5c1-0e56ad631495', '01432ff8-955a-47c5-b665-9e3b030f909d', 'f8760856-2ea4-4f89-833e-7c672a2ef54f', 'f37d617f-64a1-4560-85e7-3656703fbd18', '214bfba7-572c-4d07-afe3-1421b94efaff', '26963d6f-35f6-46e6-8a33-94add9111f7b', 'ae07ce08-1e2b-447e-806a-b5f96013bf23', '001325e4-3b30-4168-927e-590103bce85a', 'e00ef03b-af7d-40d2-b5c4-f5bf948cc372', '9c9c140b-f706-4226-bbda-9950cb3e7e0e', '3b8642de-6fce-4721-aa49-9d1d63858bbd', 'f5fdf95e-aaff-43a4-8704-88046c16f0e2', '582e5156-771e-4f49-afec-56511f46df00', 'bedbbbab-f4e5-46f9-9732-8502ac4f5810', '94a8dda8-2e9c-4a69-97ab-ba0bfcc63981', '58c8aa49-9014-43f0-bf5b-d03cbade34c0', '577785e2-d134-40fe-b9a8-9eaf978fb67b', '0a023c7a-aab7-40a3-b867-63f307a7e3f5', '85d4d26a-9348-441b-a876-7f3546795321', '40faa2ad-a6a0-4e51-bc8c-a9fd92e740da', 'dcdd5ccd-96d7-43bb-94e7-a1099f4bfcd4', 'd7206195-0162-4ee9-a6b3-04ceb279bf3a', 'a880ae15-f8b4-4fdc-9f4c-4b0d5cb86453', '77128aab-c69a-4972-bea2-90cce0209793', '8b437b23-1e6f-49fc-a7b2-c532406ff181', '7662670f-bab5-4ef5-9237-479b1e61561d', '44d8f22d-5739-45af-a4ff-362aebd76359', 'd80d761a-2e72-4f5a-bb32-0e68160fbaf8', '3c210669-e296-4518-9563-be9d25f120da', 'ff43b22c-be14-4da5-b80d-36d8c54a860c', 'a30a9427-5e37-4471-9504-44c2d979c5b4', 'e72f919a-2482-420e-9b67-7355fec0316c', 'fcaf713b-1bcf-453c-8091-8e6f34635f0d', 'e014e8c4-efec-481b-9ed2-387c349047ee', 'a50769c7-88fa-4942-9e0e-4cb3595a5c30', 'a3c3f631-daba-43e1-8dec-871740520dc1', '78fd7d31-72e5-4a21-983e-ce48c7a7b713', '238101b5-c86b-4985-a776-3e973c0b60fa', '88520078-a760-425a-ab82-3525ef1c53eb', '92c9eef4-615d-48a3-84a3-cee217f08dce', '8837c3c4-5fbf-426c-a135-efc2f8c355a7', '37199e44-9bde-45c4-98fb-f8c75b0d3a5c', 'ba366f8f-c63c-4119-b420-2c303055cf86', 'f6c2252d-2268-4bb8-bc22-19eddc7610c7', 'aca54de8-c4f7-443e-aaca-7a111e51cc28', '2a81fc84-89bf-45e3-96d5-02a41988759f', '4a52564b-9d2d-469e-ab01-f09f3cdb4bec', '82de9bdb-0c15-45d4-9661-e40f8ec643f5', '13a8ff53-507a-470b-a2f2-fa671d340f4b', 'be36ad87-707f-4732-8a4b-162f66bfff59', '51a8e61b-e2a5-4064-9a56-68018ba339f2', '3b36b955-9c40-4a67-8695-6ebbe0cda689', '306a0b9c-142f-4a95-aa86-5296dbe02cc5', '8a7bf70f-b39d-4901-b42d-15428712deb8', 'a56f9f3d-b945-4159-bef4-429464666018', '6f226f67-b184-4a1c-9718-ac5c0b72e657', '1f67154b-3464-48f3-bfec-9852b358cbdf', 'feee80c5-5ba1-497f-bf8c-7b9bc27cfda0', '9e94d5b9-a9f9-438d-bf22-ab385101ba5b', 'bf9b416e-20fd-48ff-92a5-e238ddb61bd6', '10c57cd4-3944-4147-b429-b5b89c258f97', '670a4800-7ff6-4e55-9b4f-b46b5fb428a3', 'bbe619d9-4b8c-4abd-bc40-7b4017d42fee', '0d9bfa29-bfea-4478-8096-75c79e4dacd7', '0e948d3f-0f2d-4ff5-8578-e597302ee69f', 'e89dc789-cbe5-4c13-a583-6a1301d6acf7', '6e14057c-e840-4c19-9129-61edf2345a07', 'fb8ab3f7-dba9-492a-9091-63d448aff8f2', '0a4dd387-f79f-4f2d-a835-4a5e4fb9b5c2', '0dcb73a7-8912-48d0-a92c-49d62ca12e1d', '839c3392-f7df-4dac-99aa-faaa0ffbc1f2', 'd16d3b0e-751b-4387-9f22-804830b458a3', '1796c9a3-1cf8-4a66-9371-2d52508e5700', 'cbbc9657-8f2d-46bb-862c-6adbd8262def', '3bb43de2-ba3e-4750-9f47-debf358864be', 'ec9b93bf-6a59-4346-9df1-49b056a88ddc', '8b9ec5ff-f9d2-464f-aa26-034bfdedae22', '7011e504-95ac-4051-978d-74e49a6a417b', 'af98123a-50e1-4f30-848d-b652c92345a9', '7b3da12d-92d4-412c-a65e-9a5f565e9883', '14aedcd0-7413-4ff0-921d-0dead6637d4a', '68414d1d-514f-4108-8375-22274b3f2dca', '6cad175e-7e1b-470c-9598-263e629dba58', 'e266007b-24a8-4db4-a5fb-0b1cb7d27945', '18b4bffd-56b4-4aa9-b6a0-f69907fac144', 'ffbd48a8-331c-4efc-bcbc-83d55297b0c5', 'f2e485b8-428f-4e43-8b2c-236f117503d4', '2104385d-94a7-4ae7-8fb9-47feacce37d8', 'e1b59c88-2405-4b3a-9532-73440e2f5282', '37b44564-498b-442c-b259-5368844123d4', 'f2cf7c53-362f-4b7f-b49c-1c4e4bffc040', 'cc605c9b-1ace-499d-b8d8-fdb69ae3ceea', 'e497abb5-a45c-4bc8-b91b-ac72652a730b', '1f1db2e1-ec7e-4773-8ae6-022310685050', '2be9f8fb-fcb3-439a-802b-7c35d9fe987c', '4236e321-4341-469b-b076-101e05a1c634', '0fadcb76-b0b8-4881-b50d-7cc81fe937aa', '01b1ddc1-f902-4110-b9e7-1cb93ba198e3', 'b28c27bc-96c3-4d30-8b23-29cb54f22782', '6f0b1a90-314d-4230-9491-be3fd8160b48', '765a18e9-f395-4ddf-b33a-49588400d911', '0184a453-8434-420b-b0e7-af6dafd6be94', '6a14e5db-843c-4c73-a696-14fe0c37c9a6', '70195330-0f1b-46a5-b7d9-6b6bf9e4072b', '61737029-1c60-45dc-bf45-2b89048815d0', '8fb82bec-30c2-44df-bdf5-44d93a4ca5a5', '1550bb5e-b1a4-4e47-b433-714984a346f8', 'ea5b4a58-a322-479a-8f6f-40f420ac6961', '664e7649-5be7-4789-af43-679bf958c2c4', '0b100977-6974-4db9-ae62-ac6f9bde5f10', 'c6b86557-ee5f-4293-a23d-7962c990d0fb', '2804cc2c-4f0d-48d3-9112-48e6f6f774a4', 'ce0e684d-579e-4e3f-9712-705c0dcd4311', 'dc03a2e0-07d8-4b88-acd1-086899d59d81', 'b7705396-d1cc-4e43-b888-51eb0ddc7e58', '786ef165-7fde-4078-8740-fd9959125b95', '66387184-b7a8-4a37-be9b-a395c1435289', '4d89026d-4a87-43cd-89e4-854240fe7c6c', '41057d22-19a8-4ff7-ac1d-bc20e62be8cb', '4c8da2fb-5b24-4232-be92-e283831004b0', 'bb568b7c-9ed4-4a23-a24d-826aeae5bf41', 'e3654747-6ce5-4c34-9e69-227614e33008', 'a7c899e3-1e84-40d3-9d54-5c9ff71f8c66', '2e3533c7-7ed1-4a27-90f2-244125fcd07b', '2961c826-18bf-491d-91a2-2578410a18e5', '6f41e73c-277c-4be0-b93b-716a4236d8c3', 'e6fd6dd3-caf6-4e6f-8c22-0a6b73c984ff', 'ec10ba18-ca05-4a69-aac3-cb5e549941e4', 'a6f8758f-2516-448e-b4ef-6d330700c92b', 'fffa6845-9346-4dc4-8d2e-6da0881eac27', 'bb162c56-48da-4b1e-ac87-01508b852ed5', 'b6caa5d9-eccd-42f8-874a-8431edde4e92', 'f418017b-6e72-4702-8512-db7d24311693', 'd11614a3-711c-444d-9ddb-2c5adf8bdac3', '6f97d337-3e1c-475f-bd3b-7b17823178be', 'c22400fd-9ed7-4e1a-aa18-f1d92060ea9c', '0b55b0f0-df16-4b41-81d0-2ca9c5bebf24', 'a8261488-7d47-4511-87f1-4b70c625c70e', '55b71a8c-01f1-4e61-8b26-4ad3dae6dd79', 'fb4f905b-4a17-453c-b75e-b4ce53ef2d71', '4a1ff831-4b30-4354-bfb7-c0e0b77bf90c', 'e0d50e7c-efe8-4547-9e43-cb13e88bd4a6', '1947f963-faf1-407c-9995-b29c79427cb3', 'f380c991-18a3-45aa-9613-78b861f34dfe', 'f04d980b-f7d9-4cb3-9d9e-e4e7d419f9b4', '009cb6a7-916a-4da9-a2bf-c8d7a528d2f1', 'a6a45cee-bfda-4847-836d-5b6a15bb6d61', '1456471e-5a27-44c8-97d8-9cdb7d993ef6', '73ec4e34-3f1c-47b3-865e-c49be02a1450', '889f5e8e-b65a-4b4d-b007-72804b983a88', 'dccdd215-7b22-458d-9bc3-45907211a8d0', 'ca7c53f5-cf65-4e22-add6-b32f11f73781', 'd1234b73-ab53-4889-8eec-a220eb91044a', 'cc9e4024-2a52-40ee-b508-c42b42999bdd', 'a04c855f-2aa5-4c62-83d5-439effff24df', '04b45e82-4967-45f4-9139-e8c50f261183', 'd9c01ffd-4146-45cb-9ea3-3e3965976f33', '7be99172-ff42-4033-82c4-835b0a7cee87', 'cb1cefeb-b766-4610-8d3e-19e0c163381b', 'ab12b59c-62e4-4f5f-9fa4-4fab0bf511ee', '4ae77807-7394-4fa4-b332-6eea374322c3', 'a95dc000-d4c1-4bd7-bd1f-ff8845c97940', 'f0b34f41-c6bc-46e7-be6b-3d655bf3c2fa', '9f20533d-1fd8-4673-a050-284b825543ba', '59cd27d4-8ffc-42a1-bb7a-dc2c137b1933', 'ed9d63ef-ed9b-4000-9c9f-d97486bb6313', 'f198c82e-e1b7-4eca-aaf2-7c18868f4600', 'ce40d545-6550-4aeb-a7f8-a735b53b8141', 'af954bb5-a10f-4799-a3d0-10a0d55313bd', '577cd8aa-d141-4ac8-8b74-29ff70a034a0', 'b4e81915-13ab-4309-ada8-9791e009e78c', '9641406d-76e3-44cf-953d-d62bb3cc38ea', '3fad8bbb-41d0-4ba8-8a5b-2b0d23302f5a', '600ee48b-6826-48b5-aadf-b58abd34f5fb', '9291a66a-88d9-4f40-a93e-02b5b5d3a937', '464c0160-2f24-4146-b155-ef58aa98875d', '85306219-2678-44ae-8ee6-700b28ca0c16', '69dfed45-5682-4fa8-bf65-d94bf0a958de', 'eecea5f3-9ecf-44ba-a1fb-bb0e38a8bae5', 'b78ca953-47e4-4994-b97b-d5f30cb26160', '45da1fa0-516a-41f1-beaf-7b0047938b36', 'a0000fb0-481e-407e-9a95-a78fc3f44d25', 'd2c8d550-8a95-4b31-b00e-12fdb6ac3bc7', '4d6fe570-837e-4a57-8d3d-49be88b2101e', '3d623604-6f61-4d33-8501-7ad81f063617', 'c568d725-1788-4353-9dff-531c9bae7d67', '954fa2fd-1746-4b01-ad9e-aefde97c7f30', '01dbf841-f24b-4efb-ad32-65d63130f31b', 'fb5a9539-4230-48e4-9fce-b0cd6e6ab6d0', '63e33c7c-ad25-4466-9ea5-137a5a078ef4', 'fc596c57-8872-47e1-8e7f-e8f866d86176', 'be5b657d-31ee-4a84-9250-d007fe761bef', 'fbc889a9-3e3b-4f64-a82d-4a5e945ae91a', '1cc479c9-58ac-496b-b318-42eca2b4a176', '5b08f0e5-1871-47aa-b176-6c28e431cd64', '4e2e5f41-a617-4156-8637-a2b47a5a2bba', 'd1cacce0-33c1-460a-9c13-6c9bb82d5bb7', '1438c41c-3724-446e-9470-6b6d07775655', 'ee2b287e-1426-49c1-b950-eed64810790c', 'ee9399ac-b225-4d37-9bcb-ae77e14aa39d', '254d4d15-0a7b-4020-bc2f-72258c1c8eec', '1fdc786b-94fd-4481-9215-6709736862a1', 'af1f2fce-4e16-4cb6-8e10-37d3243d2e5c', '971a0755-80ec-4c36-bfd2-3d93101a5abd', '11dd23d4-bbf9-410f-bcbb-2b9b647cf24c', '3a0ee21b-ebf4-43dc-a5fb-920833d4cce1', '5c3fece4-9b5c-4a46-99e8-2a6e7a265c7a', '55860ec2-2462-400b-95bd-99a3222c063b', '1dacba0e-f104-44f4-9b0a-f9bd23c6b6a8', '58293dc1-5c7f-429b-ba74-e9d42d1666f3', 'a00cec11-d330-4a1e-ad66-91971ce5b9a6', 'e9c089e4-563a-427c-b932-4a216df08a57', '94a34b61-b616-4caa-9c26-c3155d20d919', 'd2afd0b0-c524-4918-969f-433a9ebacbd6', '143a19a4-7516-4d81-886e-da40672df71d', '785f1ec8-1ef3-403e-b97e-4a8a678a5939', '8afd7b42-456d-48e4-883e-5c3a5d379be2', '209e8f21-7a8a-4a2b-832f-6b32d1ec0ed9', 'd25ba1a7-6e37-46ed-86ad-4f4bfb7df314', '78b4e008-49cd-470c-ae8a-774ba969b7c3', '49e89fb7-3aba-428d-8f04-160ef9bcf002', 'a22e0506-a966-4a0b-a991-bb4a8408cff2', '30db5f24-5503-4749-b8e2-821dc53792e7', '6674dba3-bb9a-49ae-84c3-a2ddd00e8ad6', 'c0603c93-df1e-40bd-8ce9-6418a127d6b0', '5666f6ff-3d69-4235-994c-452c0fae18fd', 'c948f1fb-4350-4d5d-b1df-fb3a14ac4817', '3a3e9ade-6c09-4828-8407-26f3e688d5e5', 'c906dc90-d7ca-405d-a45b-815c94a71b7e', 'fecb9d13-77f6-4006-a1bc-bf7d840d9f5c', 'fb9757bf-6622-4822-8f32-038495aadc6c', '4896d82f-b449-4189-b263-fd11ab18490d', 'c84fcc4d-1a97-4cb2-8925-4874ee6137e5', '2cac0691-5a91-4653-8f6e-74a883d2af5d', '38111bd3-4034-41e6-9ccc-782c5ae900e7', '1ffbc453-234f-4598-abeb-445d743b2d73', '589f9f17-d562-405d-807a-d06ca5f19245', 'bc7a86f5-a363-4d65-a3d1-5cf5eeb5513c', '08bdd2f9-f2af-40e5-92c1-55503fd797de', '68ff0746-0354-4f85-bd0e-e18806b5264b', '09ebdcdf-8b9b-4517-a588-00e15d641284', 'b965b829-c919-45d7-8f57-bf91f7115ff4', 'e16906cf-be6a-4cd2-a595-03325aa3f6ac', 'd6b8dcbb-e14f-4f11-bb76-f75033b893f8', '9f12fd63-092b-4cf3-8856-28beeef3485e', 'db061e8f-981d-4327-af00-5df74d7c5d30', '5ffed754-cb53-48b6-b90e-7a84d20cbcff', 'e1836ab1-010a-4a7d-b4e4-17caa2e57c69', 'e9d3bb15-9510-4ecb-9944-7b670002f3db', '4aca9e0d-b421-4d5c-97b8-1ff1c05076ac', '8ac9c40e-78a9-4ada-ae66-4da570b5cd9e', '24153d44-a4e5-4967-970c-02cc1409cfc4', 'a2c2c5b1-61d6-480e-987c-36b99a885833', 'faa09b75-06b1-4c1e-923e-770d142e3387', 'fedc0335-bfdb-431c-9ebe-2d0c5dcca610', 'e817d23f-de35-451c-8fd2-ca9c50526fc0', 'a0e22a54-72bd-41e3-9f5a-2fb084d17ddb', '6a3f10a2-a6b1-4d76-934c-dd426e139ba0', '56dbb27f-1ce7-404f-8a52-a85fdcc16099', '42659b80-961d-4ac8-8dd1-c8abdd91e151', 'ffdaaf16-0c6b-4a84-8f94-ec37d83fbf93', '8a00232b-280a-49aa-96cc-4db769877bdd', '770d3355-bdad-40a7-9165-8c87037ccff9', '43da1381-ceee-4b61-9744-379c9888eebd', 'f73e1b31-a070-4800-9257-1b25323936c5', 'c44bf0b3-e403-433b-90da-1f246c6f80f2', 'b1aa9247-ff1b-4b54-8d12-128f806a3983', '69ff7038-e436-4385-9ace-76489f5274a4', '9c9b2b37-25f3-4ea4-9705-88a83120332f', 'fe26cac7-fdcc-4383-a30c-d8de8a4d3c21', '16bf5aeb-22b7-4088-8df3-2ec0f278a90c', '60b683d5-e141-4352-93cc-20bcdd311f74', 'c424aa8b-f99b-43d9-8517-385e52405670', '35a53a62-2092-428a-af42-c36aa4a2e4e1', '1d5b1c86-5db9-4624-8257-59f524e8ce7b', 'ccf33da8-9d54-45f0-b267-2705cf6a4186', '7dc11649-1c7d-4afc-ae36-afbc6a0eb062', 'e0fa8e7f-6095-4051-971f-3d56e8b47ee8', 'a43d51e2-fcce-4c08-944b-cf9ce5083ab7', 'e3450d94-5f06-487c-8f1c-328bc6fba19e', 'b1797cf8-cb1f-4ebb-ac16-6ed10106b9ff', '3af8c6cd-f2d4-4c35-8e15-38a6e8156123', 'dcf5b496-a31a-4753-968e-77837002ca21', 'fff09ab7-441d-4a0f-ae90-674a718ec4f3', 'a4f9530d-abf9-4fd5-b392-1c74596d99d7', 'fa8397df-4f59-44b5-9c06-5d5820e0bd7a', '964275dd-587b-4d13-a15c-5e2e765dc173', '8238829c-bcc9-47ae-8b4a-f56da8d9bc6b', '662725bd-a5e5-456f-b838-563b326e3a69', 'f7cb0d08-2713-4610-ba93-1d22dd5b1e0a', '8f3dd00b-3728-4286-be1f-f6049b5ede30', 'd36e049d-d9b8-4376-97f5-c845ff70dedd', '2c2b4845-5027-4121-b64e-4d71a1abc374', 'cd57e4da-789f-4a64-bbf7-ce1d90673fc9', '5b92549a-ed5c-49bd-b865-19eaa2482b09', 'b6bd256b-05fb-4b80-8c50-7ec02c917143', '346f57b0-657d-4301-97c8-44b7290e6430', 'faf7e41c-e9f5-404b-9221-0c6b012ee740', '3d87c3d1-4f69-4fcb-8b9d-da1f031cdf07', 'cec17d99-408e-4595-9216-0339c1b1a12f', 'c4bcdb1b-9690-4b93-a9b8-55e2e3a3ea9b', '8121b060-9737-4559-b17f-75f227ac9048', 'c4de4049-5c54-449f-94db-e624dc87ed63', '7a5788cc-73ae-44da-b85e-b25ef06234e3', '3a74b557-b525-4dc0-a0e7-e14d50c91d0a', 'cdb1148a-23a6-4b57-b907-03ffc91195a8', '823e793f-a8d3-40fe-be20-df3b5ad9179b', '5aa069f2-bf6c-4c7c-8fec-a00897397e56', '1cf9d498-da4a-4180-94d2-0d5c00afe650', '6aac90cc-9579-460d-b2ed-04a81c109de8', 'e544e2a3-735d-4503-afc2-47d009f7935f', '7b08d4dd-a0b4-43cc-9d2a-972b135c1430', 'c64b8f84-967c-4071-b291-34ba48437ccf', 'fbaf7512-2908-4177-8ed2-42a865d610d6', '62753820-42fb-40da-b36e-a401f377357a', '6a3758ec-79fc-48ca-aac0-e30d12d3b6d9', '3ca70d72-b4f9-41e4-b92c-8c029e1cd2d7', 'a65dbb38-6e75-41df-ba67-0e189e721981', 'ce9c7edd-74b1-4fc3-b808-b611a0c61df5', 'f7b396d3-bdfa-4346-b07f-e6b751236aad', 'f3778041-429e-4499-9e3a-0a0ec8f16936', '165d33c2-263b-401d-91df-0117a29be480', '68fd8c46-2475-4e55-a9b3-42ee20771cdc', '945fb4ab-2407-4ec6-95ab-d9be92246a1e', '6c4fda67-9ddc-4459-afa0-be683844dd0e', '25c8a813-432c-476f-aeaf-b73c7efc2337', '711a3536-173b-45d0-9b9a-4d98244ffc01', 'ce1fd1a6-558d-4261-bc5a-b9b1d3568adf', 'f8132790-8261-4615-82bc-c992a0dc4d0c', '92026d5c-2af4-4b91-9583-fc592f5f71c0', '1bf44a11-d803-436f-87d8-597ce52acd11', '388fd340-e59b-4d1c-9db4-11b4ca6a731b', '609deb9b-c3df-44cf-a011-5b0ae760bf55', '7f558621-b3d8-43f9-b03e-954ad5175cec', '9e35f44e-6f91-4f66-8b91-436d0a57466e', '97e79043-346f-442f-94a1-fe8e792245a1', 'a44385e7-cb47-4f8d-9530-d060cdbb30db', '3dd464bf-f81b-4f06-91f9-9cb3c015b4a3', '1284ecfe-4f71-4dff-a589-8ae7cec46a06', '4649c51c-cca0-4694-973a-c9be70500ecd', 'eac9fec0-2641-4426-8f8b-3a48a0b32821', 'ad07e91e-1c49-48ce-9ae1-335ee19c8982', '1446656e-6f64-47a3-8a54-95b85571d145', 'f1bf0c6b-53a3-4503-811c-e63935f6cedb', '623f3cf6-106a-4a41-b707-92839c6859e1', '17f1a49c-29d8-4874-b0ef-5dc9a69026b1', '73f26647-937f-4b0e-baf7-5f7d891ab022', '6566d6a4-8755-4145-9c53-c66b367852fd', 'acfd2e4a-e422-4863-b861-bd4c1cbfab87', '27eca136-22f1-4f3b-a5f4-47f2e3f5ef28', 'ba02ed24-3ea0-4d76-ad58-055b1ca60908', '1502208c-631a-4cc1-bd71-30d4da929264', 'd72bc675-a968-4981-ae8d-4b8104a8fc94', 'a41600f1-eaf3-467d-bda1-3490acc6c6ee', 'b6829360-ebaa-4356-b51b-2856c130a04b', '95e4fb59-1a70-4dc6-b2e1-3a1b5cd6f0cb', '5e85b3c2-afd2-486a-8324-3da82026e742', '584726a4-ee3c-4cfd-873b-5c09940aace4', '4da78a20-e58e-44ef-90b3-719d970e3608', '5452a771-253f-4a3e-a7d6-3878af982983', '8be25d2e-54f2-45ad-aec4-f0c41d0c6960', '8e6b865b-b83a-4ec0-9137-268a6bbf2d46', 'eed75b04-4032-4491-8700-12abb6192242', 'e18fb7e2-a107-49e1-8559-c8a453a0c12c', 'f1e12046-5ff0-48da-978b-5054e910d3d5', '4e7b5bb8-5dc9-44ba-bd86-01b2fcca4f35', 'e065b96f-c7a7-4109-9a98-0abc08279f41', '6fc1a1f7-48c0-466d-a902-be41bb2909dc', 'c3558061-f684-4797-b74c-55e8de4a383d', '279e5a6f-86ae-47dc-934a-ae06083151a2', '84b96d48-bc75-4aaf-8339-ae9a2bae6798', 'efc90b59-f1dd-4eea-99f9-3b9a58b88483', 'd140e637-4054-4d2f-9545-669a57df1ea8', '3db62012-d8bc-4017-ba31-98fb2b592936', '3e024ad4-7bc8-4446-a722-e846d3e48e0b', 'ede7815e-f2b4-4e92-a8d4-e7f530e83c70', 'f3ef8423-bbec-4546-8d54-c4d2d9eefb3d', 'c1939d81-71d8-402a-a72d-eed70e81ec98', '1af34663-8243-424b-b749-35aaff17c778', '9a21c4b4-f54d-4459-96c3-c9e6ae9e8e31', 'd421d774-ddc0-4979-a5a6-cef76679e898', 'a053d09b-356c-4582-8595-542852913eaf', 'bfc6b91c-e131-47a7-81fa-88e5eb3c5fff', '241e9854-bde0-4e7c-be1c-dea37a2f0e90', '4ca6f1e9-0c4e-474e-a1cb-d41125407f0d', 'd4253539-4006-4dc4-a613-30ebad87735b', '40da80a4-7fc6-4866-a0e7-cc1154336ea6', 'e2c554bb-3bf6-479a-983b-5e58478ab704', 'b8416495-ecf8-4e60-816a-0db10f2c5d2e', '1230ed50-7883-4176-9792-c60e73edc163', '2c269fc8-c778-4161-8be5-10533e66ac9e', 'c647a34a-44bf-41fe-a35f-1481ecc31b22', '0bcc9b1e-9da3-455a-a32e-4c1324c91bd7', '09e4d63a-c555-45b0-8879-05ed1216b496', '77b2202a-fd3e-412c-8aa7-ffc8b0cf8f50', 'fdae3cbf-c8b9-47ad-b362-9e1cdcc23b87', '4af1a18a-6733-468e-8cac-856f966abb42', 'c0996d40-8bae-4a49-965c-513c1adb118c', 'c20809a9-47ae-4379-ac6e-d799c88889da', '45a5677c-96a7-4a37-89d6-9b7035f0ab14', 'fa9bcefb-95e5-4595-89e6-77d7b9e49f82', '9ad76aeb-e965-4d2a-a9de-92f04fb9b8e4', '79f4e8e4-d10b-411c-b47f-7969eec84600', '7597ca1e-0b70-4a71-bb82-922c1a9b52a2', '608073a3-82f6-4577-b38b-71ede8712317', 'f42f230f-d6ce-464e-8025-66a23d253f1a', '02da5604-ce37-4d80-b7c0-dad352952562', '8829a113-df8c-4cc5-8530-5b9d1f318fd8', '6af90a39-eb33-40d8-a4a4-a656b94df580', 'c639d0f5-bba5-4c41-bbc6-c92ce0a01206', '332bb205-cae2-4814-85b5-1225b0d42faa', '757c7c0d-a608-4026-b7b2-8764639b83c7', '9bfa128c-a182-43b5-b5e4-ccccbe0be779', 'fe982f5e-8999-46cc-8ae2-5c0610479a91', 'c908b30c-f7bc-48e7-9b07-5f652939eb79', '289b8cb5-827b-4b96-b41c-146d1d612207', 'f00d8ffa-e5ec-4126-8232-b3e64acef8a1', '7a58b996-fabf-44f8-abb4-3ec0d3dbfeaf', '49e061d2-9378-4f1b-9816-71de3f03f01a', '29571673-755e-4d4c-9728-1abff3f8fd67', '6c0987df-8e1e-411c-b19d-adf095a1050f', 'adb1a1ad-e570-4fdf-9488-538bde8d32f5', 'ab612c9d-05e2-4ab1-a2d4-9146bd0bb6d4', 'c4a645d6-4eb7-4201-99ae-fe3faa482744', '1e051928-63f4-4795-9722-b10db743f215', '37dc3b34-d2d1-4a5a-8b3c-eb401e782bdb', 'bc493fef-5f41-4377-8a24-001b00f280e7', '0725ccf8-beed-4f7f-b741-fc54ff7e2ebd', 'f2f1f0e6-1821-4054-bdef-4f3c76348178', 'e911664c-510f-4c67-8ff6-d67968070e84', '218da72c-3ddd-423e-86f7-d0aed4bd1e47', '5e33dc80-b973-4d0d-ab5a-c6a45198254b', 'ca441aa2-6796-4db8-960b-a7977e434d43', '58a6fe6d-8bef-4419-95cd-9fc604bdc421', '8365d17b-d86d-43e0-852e-f3fda8eae2d8', 'e594aff5-132c-4d9a-952e-1fc74d243a12', '958c0b6a-6f8a-4f6c-8406-f556f75af78a', 'e40a313f-2d65-4606-90be-d168ff24a3f7', '4d21ad8f-4e3b-49a4-96fe-8f66364a0d0c', '1abbf587-b255-4721-b8f0-2b96dc32a646', '1185d64d-2d10-4e81-b958-ee93bab94ae5', '7cd32ef2-1ef1-4f42-84b4-1cdcb23ab30a', '56de7b18-ad03-4d6d-bfb4-aea2c90a6d47', '08f8156c-4637-4906-b6b2-fcaec641b75a', 'a3da1b8c-c49b-466e-b18d-613f8ddc3b05', '34a104f7-6762-4110-ae85-4f192a1db9a5', '92779f0d-dd69-4016-beee-33e8705804a9', '1f9b0305-e137-447b-b185-53a69184efd6', 'b49295ac-1d35-4294-bba8-6aef3b287be6', 'fba18575-c2d9-41e3-83ba-ec8fcf23c984', '79fe113b-9305-40b6-91b2-c9f6abd84439', 'd3ad96e0-1d90-49e8-af43-e3d38e6f7128', 'dbaec471-51bf-4ec1-9822-7a92e9f1b77e', '93dae16f-4011-4ff2-a6d1-c013cf6dbd77', '2c8fc7c8-df5a-4914-b0d7-83b71000b835', '5e83a431-918f-463a-966c-6b402e6021cb', '1be0f48f-0825-45a4-9b3a-913bf013e327', '6838c4a5-f996-4a37-b8a2-15e2a0a4ffcb', '2f8ba902-14bb-4b79-a17f-aeff5c95c84a', '8ecfd421-bd58-45b9-b323-979d6e3630b0', '580aa329-04e3-45e2-8acb-fe14cb26013f', '04e49400-b1fd-43a1-a29b-b2b7deffb80d', '131fa278-c4ed-4af0-b2f1-1fb805dc2a51', 'f8530d7d-3d78-4c4d-a62c-e627e9d28065', '87e1bac4-6dfd-4dd2-ba6c-bf8a90c5453a', '4a26f80f-e99b-40a7-af80-ddd13ab9d582', '6fffeb0e-5192-4855-b76c-325a13629478', 'b65e9824-88c2-4a62-aaef-e317a555dff3', '1244147d-e91f-4a43-9903-ec672032953a', 'a3e59377-ee7a-4002-a95b-02dee37cae6f', 'e261ceb3-7a43-41a8-b870-105307106d6c', 'dbea1d53-4616-40a5-b389-56118fc689c3', '0ecd2cdd-7394-4aae-935f-083440f44c73', 'f830a09d-6403-46af-b959-8c9068eccf11', '85e6e031-95fc-4ce2-9737-b9b457a11759', 'e42b080f-17cf-42de-9fde-5a75f5d6da7f', '28e519e8-50db-407f-afc0-b7bb0f37ecbb', '066a337d-30d8-4291-a38c-3c98191037df', 'f29fc06c-9bea-4105-8982-01646af59e1e', '28dd8e74-e4d3-49fe-9918-84cc4ec0f499', 'ee3929a9-b431-4bc9-9334-fd309a38f152', 'd664efb9-22b5-4dc3-b65c-0b8ac6b52c2e', '3398388f-e461-4c4f-8673-88cd40aee29e', '358a54e2-b53b-44b4-98a4-d301e8b0d87c', '583533dc-a2d6-44dc-a848-e70aad3c5ee8', '11623846-afc1-4b5d-a7a8-70cf65f99d71', 'b174a6c2-747b-41d1-b7fe-88d2ef3b94fb', '2f50427d-088e-47a2-a79f-c9f02938e184', 'f4b13771-8208-4490-9e07-10bb00f4ec2d', '74960b3c-125c-469d-b324-99edb8902762', 'a0ff327a-4777-4ea5-a473-953e99f48fce', '1d6fe346-cbba-4f0e-ad78-ce674e51562e', '21882b2c-8f85-4025-a05e-9d5ac5c7dfe5', '8676b965-8b56-49ab-b6a9-a54292a4c53f', '022cc406-b4f2-420a-b707-82b789638d65', '712394e2-8cde-4fc1-828f-1ea228fb85a6', '3f0f56ed-0942-4f28-b2ab-e015705a388d', '1ff1ff9e-a535-450b-b5e5-2b3aad2a92e1', 'ee0feec7-0fc2-4f57-8eea-f3d9e2999db0', '11782be5-5a4f-4d91-a1f7-66a85f90ce4f', 'd3681b6e-accf-4e7a-8599-9732dff3b868', '9809779f-51b0-45e9-b693-edca5cfa39d2', 'b9b299b3-6398-452e-9651-6189a49e5fea', '010843b8-b1a4-4ece-95e1-d8430a677f49', '7300c61a-eec4-49e5-a60b-1ac573d1c2ca', '3e8314ac-6de4-4fe0-b3b8-8090e7238d54', '53437029-9e9d-4417-8d26-d440ce43ab8c', 'd3a2479e-d6f6-4769-ab79-f8f399872ac2', '3dc7a690-4000-4337-9052-3c175d5b17d1', '984cf644-5297-4ea0-8e48-171a2b9c84c1', 'ed944c6d-d921-486d-96e8-2d616bba357c', 'fca489be-e88c-4540-b01f-3eaa4cdc376e', '777fabc7-034e-42c9-a21b-bd8346081baf', '2e646167-2df2-497e-a3a3-3c3ed4e65936', 'a68a8b48-b54e-4fda-b0fe-f939c87763ad', 'dfc6871c-16b2-426b-8d7e-f339a06594ac', '63047547-b550-47c9-b993-2e886acbf560', '5812319e-e240-4a68-ae1c-eff464ba0a7e', '7e185c0b-5d07-4465-b049-bd18fc2048c9', 'acc316ea-6132-40a7-bdaa-c82630a45736', '9faf5a46-083c-4779-a318-0a209d1bfff9', '1c25d563-9932-41d3-acba-b0ca5aefe8c0', '12afa242-3b02-471b-92ae-01a7a666d975', 'd27cb9f0-260e-457e-811f-ffd1448009cc', 'e638a2b9-395d-46aa-9f48-ad893802cb22', 'e16d1536-3624-47ea-a0b2-4be9049977d5', 'c9ef6859-d2a4-4586-8477-efafa7d2937f', '048381bd-f60c-40c8-85c9-867a04ffe5b5', 'd944bff4-eef4-466d-abc4-a0323e8486f7', '6baa86ac-ebc5-4808-94cb-30af7cca84ac', 'fa71937f-fc79-464b-ac33-3f84913bc5c1', '79cba46b-3a56-43d4-a711-ed1c27f8a8ac', '2ef579a7-8663-40f0-aef4-ba3069611485', '310f4a54-46a9-43e9-b8e2-df926bc1fc13', 'f61b7271-7b94-43b8-b404-a9037f54d120', 'ccd2e48d-222e-499a-8e5f-ab3d417b96c8', '5377676b-7e17-43de-9646-f00a238b562d', 'c824be34-b6bc-4635-aebb-4714244319bf', '3b813add-6d07-48bf-b3f3-300ea3fb0c85', 'd790a0e2-32f6-48b7-9c6e-dee360e54606', 'b6f88864-38f3-4ce4-843d-1807f88ec790', '56f5050a-0fee-4e03-a7fb-334fc44234f6', '186a9c80-281c-4fda-acd9-271fafd649ad', '8eab6bde-c670-45bd-b91e-094355c25bb7', 'c3abbce1-9d00-40a5-9d38-2aff1aed5129', '9bf93fdd-2fad-4c46-a7a3-896d816da7ac', '28d55ac5-c96d-4867-970a-5a69a0bfa928', 'e4d7cf20-4a9b-46e3-a36b-4dd1d1ce97da', '6dc0566e-ead8-43a5-be56-dec682c7c63f', 'f96e429c-b74c-411b-87f9-eba9d50e53d1', '82879c65-a05d-448b-b0f9-6a7083ed4590', 'cdd34ae5-533a-49a0-ae1b-2dcab7087683', '2effa546-1ce6-449c-b3d7-0564561ec4ac', '8fd62893-b488-41fd-96c5-61a7ff5af1bd', '440ee2f9-d226-4f4f-bde1-d14200833b29', '011229d4-5e90-407c-a670-49ffc98f79d1', '5cb60160-7c7d-4b59-9203-2c5ab07842f1', '65a29211-e00c-439d-a26b-bbb119eb9630', '4c8ba24f-6d56-4d7b-91d9-68aae2ae8f74', 'd3181ea2-2e2e-4151-82a0-6dd3bf9aabba', '1bd6158a-c5e7-47ce-b0f5-6ffdb35d17de', '8ad9e821-9357-4562-aaf1-7bfd3b93fb20', 'ead39029-9ec0-40ce-9bad-b260809b8ced', 'f77386bf-9902-4005-93b1-73c0f9f397bb', 'cde94ddc-3f20-4c89-8a3c-9bbb54d26a99', '78758cbd-9265-478d-92d9-b7cccd2f4751', '8687f805-b0a3-4260-8d2e-3c87b697ce0c', '99a5ca37-83e4-4e3a-9187-587f8a2caf20', '6f9ba684-3830-44cc-8ed9-29e1389d71be', '9dd5a52a-3035-4902-a140-1b44b469fb5c', '697e286d-b7ad-4447-9b92-81fcd59c3236', '60d17e33-b6ec-4501-8063-2b3951911a42', '8af0aa51-01ab-4f51-82c2-9d4649f04066', 'c2b5a36b-d306-4db1-a57a-bfc5066833ff', 'ace9ff04-b28a-44ee-9eca-c9822f577fcd', '047eea06-4eb4-41bf-8874-8bea4d8e056b', 'd7227627-de83-4167-b4e9-1360bb1c99a8', '761e0a99-e835-47b3-adde-5cfcc0d6e48b', 'fbcc04fa-12f2-4242-bdea-9c53ad092f13', 'ce4707a7-28e8-41c7-a022-0d4a7b2a02e8', '1f945598-5c9e-4cb4-802a-74d8682d8cd2', 'ee83cf79-3a92-4dc3-8a01-38c8d2817290', '17deaf2b-ca98-4873-bf41-35e0b79bcddd', '022dee22-57bc-48a6-aec2-307aee481dc8', 'a338029c-77c2-48cb-aaf9-d6708dfcc628', '3b9eb7b6-bc65-4476-bde0-75f07d1a479c', 'cb646236-e11c-4a30-a4a6-b40305062d2c', 'b7d15d9c-e206-4355-bc66-600e5727a2db', 'dc274eba-cdfa-493a-8b22-6e418157eca8', '04ce3382-304c-4c10-b79b-a288e40525d6', '9fcbfee9-0f98-447c-9669-0bc475f8554f', '9df3dcd2-7333-4940-abc6-2902bec425b6', '74965ae7-000a-4b65-8ae0-28821722a4c5', 'c0ea3df6-66d6-494c-af93-599cc8005459', 'a099661f-0f96-4b74-baed-ee0ab9870f16', '37f6587f-2a6a-4021-b5c9-e38c3842e331', '83ba575d-c051-48d8-ab84-0e7a4a0397a8', '3da14946-2583-495e-bc5a-eef658d15c95', '6a51f7e6-7131-46a5-8fd2-a4347e699df6', '4b3860e4-68b3-4a9d-bb0f-c79499310593', '5333c492-671a-4354-9d2f-7d73251f1c39', 'ea7a55e5-3749-40aa-84e0-9738398e69ae', '4adec779-4085-43f9-9749-a4f79f5519eb', 'bdf82a38-e0e8-4f61-a3a9-ab71dd33fec3', 'c2fabf99-bf05-4453-8f83-cc8eca721e65', 'c8670329-b9f2-4472-95b3-14552103e125', '4e7ba94f-e9c2-4941-b6ce-383d82c0f65b', '189500e5-532b-48d1-88cc-579b774e8f7f', '07029ca3-e0b5-4d9d-81c7-22db124dba19', '5dc731b6-a1b1-458f-be71-b41cca193acf', '7d9a6280-47ee-4957-8405-1dcc209613bb', 'd97a3c4c-df2d-4051-8c51-b3078efc48d5', '5715f1ac-2a29-4b6c-97d3-8fa5ae66c115', 'f7d16528-a3c8-4c12-a34b-1b98d3e2b218', '4d45ca7a-94ec-4d4e-8f32-20a567f835e1', 'a9d36583-5e55-4934-9f8e-d8d22497ad27', 'ca83cf8c-07f7-4246-9428-991ce4845a38', '2d918405-702e-48ab-bc40-3dc1da4a7a2e', '86a210bd-92f9-414a-a467-2a842cfadaa9', '80eff49b-b490-4df3-a771-4b0bdae7592b', 'cf4ed4b2-dbba-40f2-abe6-19802aadf541', 'eba3010a-a026-4e9e-94a8-ae0f77724cb1', 'a8068f5c-6f96-4bff-b79d-a3dcfeeaa280', '0abdc94f-1499-4384-b38e-c8823b487f21', '928bfa94-b605-4714-b124-5a25c5e42bd8', '7ed3d189-3a99-4816-a1c7-5678c119d8ba', 'cb30f59e-dd28-4a75-94a0-259942493875', 'dff956e7-f33b-4c03-ae57-9538dd622b91', '8bf15a9a-69cf-4b50-b60a-514b36ae96df', '8f174427-d4fb-4f4f-a59f-5e0345bdadd9', 'a17c90cf-7203-4d92-b33e-0f3831ffe006', 'fc182a83-5472-4e2a-a9dc-7e0012e483c5', 'c7afb8e0-c173-4c1f-ae90-a1c65093017b', '41613cc1-c095-4291-869a-df02557c715c', '7e7eb2c0-d1f3-4347-827b-0acfce995056', '84ab8378-05a9-4ffc-af1b-e77135596962', '733e3537-1785-4484-b0cc-33df13195ad2', '59f24c01-a820-46f9-9ec5-26ccc2bb9b99', '22a79425-10fb-432e-a949-01a37c22c495', 'd40ca857-6493-444f-92a6-52947467ad7c', '02d1f251-1551-4f4b-8ace-5a89a897fba3', '97aae706-e442-4648-a616-eb9daf306a4f', '9969d379-361d-4a47-a876-150d83e0b7ba', '802ebbfe-ccd9-458a-93a4-48509d998992', '5eeb1e61-0ea5-4f40-b98a-4169bde6fa30', '124df5fb-bccf-4daa-847f-7576261e09eb', 'c5e60296-9c4e-4eb9-9df2-26a6d27ac31b', '597170dc-c5e5-40e2-9686-a29391e2f08b', 'fe1c3779-aa50-4fd7-9794-23f1b10f447e', '3851d527-deb9-4e67-9ca9-46a76985faeb', '0902ee22-e88c-4f0e-85b9-5e83fe2f0b30', '4efb2bd4-7b01-43ab-8d5f-b80aac7a71a7', '19ae0fce-1d97-4bd9-95fa-aec4f8cebc52', '6d90d74e-28ed-4911-95e8-51ac37233027', '49e33507-c0b0-4bd7-8209-f6ce0dbf15d0', '0dbe9eea-5849-421d-a500-6cd545a2c12f', 'ec2ed287-28c1-48dd-893b-c8cda0a508b4', 'd51bc6f2-1e50-4673-8e60-eae257787176', 'b5cc1352-4268-4513-a03c-bf633544e2dd', '264dbc8a-7fa1-4220-8d5d-c008824f5870', 'abac75c2-3ed7-47af-8df4-2c1597b67964', '5fe05a90-e94c-4c88-a16f-d82799a9bef0', 'fb50169e-785d-4403-9853-adffc6b5171e', 'e70c7461-4be5-437d-b243-abf069ac3ae6', '6f561a81-b15f-43ae-9cc5-eeb0f9a38a71', '1503fb5a-3fe8-4bf3-9eb2-7a903286756d', '0f3d6f25-b150-4f89-8f19-47e05112e968', '7ee44e78-1c9c-45ca-9757-b59f1531492e', '6aeada42-ec65-4bdf-8831-113036a29ffa', '49cebfb6-383f-4c28-9b4e-9a8c875ca489', '3d817822-730f-4917-9401-22a2a406fe8e', '7d054de9-59a2-449a-83f0-aae1c12fd104', '2b1d43d1-9bc8-4fe7-9f1a-04f6c98097ff', 'a44e1a27-2f77-4160-825f-63e89fc2e3a4', '29d3dc08-b4fd-4b56-815b-5f9fa7d351e4', '7fcfac3d-00eb-4bb6-a342-5833b1f3037c', '02a71b1b-d156-46fd-b8e0-919d3e68ab67', 'e2d02080-838f-441f-ae03-5b4e77cde9c1', 'f7c506e4-0ce6-4792-89cc-3ffd47246bec', '69e19bec-8ad6-406b-80cf-51ab5d80b65f', '37ba87ed-b988-42ab-b71f-19a26582f05f', 'eca7eac3-64d5-4db8-aac1-7d2171b5b594', 'e010414f-d677-4efe-99da-b7b66755d4ca', '6bfb0168-5032-40ec-a757-5f9ceeee0c65', 'e2d76a70-0ae1-40b6-a610-899a66e92716', 'ec03565e-b463-4894-a441-f6eaa1e14497', '1fe06423-042e-413e-aefb-31ea7173c770', '7c4d266b-9ceb-4335-943f-3decf874437f', '09c24c3a-991e-4e6c-86dd-00775ddc9db8', '1a1f374c-636c-4ffd-bba9-795be4d7c2f2', '663a0264-041d-4bed-b616-6e20aeb74b39', 'f43951b3-e42d-409a-aa7e-19dff1102007', '9d22157f-08d5-4c08-9fab-28d2e2ae7b57', 'c6ee3756-e8bb-44c9-b0bb-119e9f03b360', '34bf7959-f4bb-401d-b934-7920d274a088', 'a7616d94-b3fc-42e2-98b2-03bd04c97d0f', 'd312af10-76f9-43c3-9118-db3f81bde31d', '48c74bc3-3e15-4ba7-a5ac-786074cab253', '3e87029f-6dd3-4cbf-8d6e-a4f83bc8bc19', '9c49fa14-3e11-4b12-af33-d1398946ffc3', '9985ed29-19b2-4ca2-a114-d4a5db55a349', 'cf2c4e64-dad1-46be-befd-f8dcc859abcd', '8d96ae9f-9dff-4863-abf3-283f80f1027e', 'ee3322a3-875e-4848-9aa9-bfe069f0ae65', '9e62b556-144d-4381-b9f4-c58077f08685', 'e1b38e4c-4612-4d4e-9489-9953af45bea0', 'e191de29-8260-484b-a6a8-4d5ac2d0faca', 'f41d25da-3599-422e-bc46-f1b0d4c38eba', 'bbf86abc-e599-4bae-b673-973dd4c77365', 'e782a415-6fb6-40c7-b28d-d91e58ed44c5', 'b20561db-0067-42ad-8721-9326a247ffcb', '4481d451-0060-4cd0-87ed-b9a00dee1243', '7f811a3f-c3c9-4981-aecb-6bee8144bc66', '141d3859-76fd-4cb7-b64a-8e89bb6105e2', 'ad19bfc9-a815-4cce-a18f-1e50273a4940', '29ba3db3-4c1c-48c2-9785-aa587880a1d7', 'd534f8cf-207f-4b9c-b3e7-63e5396c976b', '91066cd2-7197-4c51-9c8e-3ac6a7f8685f', '55ae8638-cc22-424d-bc8e-304f641175e1', '21ddf401-722a-4898-b34f-e3dbc81ef122', 'aaf28ac3-74a6-4b67-9d27-2fb8b180f9e3', '5768f84d-bc45-479d-825a-5974920907ee', '1d3ba8be-748f-4508-83ad-056e89ecd39f', '70e9491a-e894-439c-b190-469564028fee', 'fe5e2980-4176-4c9a-8e8b-9cd47c9f7e07', '2d528d3a-e365-4f26-8393-65491081b279', 'bfaa69dd-d009-4377-be48-167890cd0179', '1ecf7494-7210-436d-b926-9e873e5b6529', '0ae7bd49-a7c4-4a09-8011-f70d483760f4', '95cf7921-2ddd-4b0a-b393-82942167c148', '8220e6e0-1274-4067-9362-d1843efa59d4', '2664d12b-7727-4ba7-bb20-6d36d52f12f0', '70edd117-06a0-4f49-bf35-29f4c7de3064', 'ccf74d3d-94c4-4ec0-8f39-eed826c566f5', 'ef765d30-9529-409a-927d-f25515bf1710', 'b3b126c8-2a60-4802-bbaf-1b098e51ecce', 'd015e83c-3cad-4f5e-bd30-4dbe071befaf', '8986e901-9e9b-4ed2-b31d-f2176a202a62', 'e0e50c73-fc45-4794-99e6-aa137290a4e1', '7572b39c-6969-487d-be36-ca0a69fdcb8e', 'af434786-c3ee-46f6-8d10-acc5ec0bec29', 'ba575443-b4fb-48f1-a008-c3aea0628271', '5dfa1861-20e7-4866-b924-cdf621dd0a88', '548d42ba-3381-4893-a39f-2237d2752a23', '9c6e8871-295d-461f-911e-2a16aa881389', 'b4b49aea-8d72-4ee3-81d5-398dd7085be2', 'eb27e4a2-d332-4a29-abcc-6ca01d089d06', 'f4f4208f-01e3-4f3a-9135-4574a328dcee', 'fd1cddde-992f-488c-b63e-eafcae157326', '9e561b7e-ec42-4d1d-9695-d93010948fa2', 'b5b4a96c-c1b6-497e-8191-a4b3f0bc39be', '51b22c8f-efd3-4377-8141-3b16d0476e5d', 'b25ae027-ca8b-4a81-bf78-94ec4c0d66ab', '34b97a40-57f8-4326-bcea-73dd6f2309a0', '7a3f1db2-3f97-4d58-b359-59fb7cd9ca49', '48df3a64-b065-4d84-ad45-92911f48d13e', '8b648bb5-7a71-47ce-811c-2049fdc0273e', '0676c759-6a02-4f46-916f-8bd3c99563c3', 'fe74f926-5f68-454e-a893-149be82f3346', '457f3c6f-f4ec-4749-97f9-117dfc06d4db', '5a728d3c-d769-4afd-853d-9d2552d239a5', '86a1e722-f662-43fc-bd17-037bd6123f06', 'ce3871ad-3efb-4785-a306-e49923889613', 'd1f00b2a-1df6-4873-b29e-3aa53818d384', '1322bff5-1e46-471b-a79b-e5da27677247', '1185d178-8e44-47bd-b4dd-a296e2bf9031', '3d4e3921-e785-454f-8ff1-f625205a8808', '23896746-3201-469b-bdd7-a0c2d3f396c7', 'c2e5ea10-1af1-4814-9cd8-ec569c0fb0de', '447eb45f-4d31-4eb9-98a3-cb5c05fbe1e8', '2e4464aa-97ee-4116-8c9d-b9c99123e0a1', 'e874d62e-9f2c-47d0-b5bf-84a9765b48e5', '1bebebdf-5387-4a03-b616-d4766e9e0630', '1b49a31b-1828-4674-9db4-1823b2c9d607', '31dd60d6-8b0e-4abc-81a3-e7bf5aee1cb6', 'f94f9ada-e246-472f-8fea-233c1b8bbd47', '24a7e24e-8902-4872-acc4-aa3665594333', 'fb4837ae-9d8f-4e78-8039-45d62225e224', 'e0add8c2-9f3d-4557-8317-3fd19c913636', 'd1459ed9-6cbf-4387-ad59-972de8282692', '5c026cf8-d0c0-4784-8058-3a035144b3da', '2d157506-234d-4988-8bd7-88e7c35d19b0', '47420f4c-9e29-401d-a29a-1341d1127262', 'd43ee8ea-6cca-4f7f-9bce-a3a8e5e7dab4', '2645464c-d1cb-4d10-9075-17315791f00a', 'bb97eb22-25db-4f75-88a5-abf0eb3c8af6', '35bf7f9c-a62d-41f5-a3ba-99402d1f1eab', 'c4f807a4-41b6-480e-b973-86ff7e73ab21', '22138115-c9b8-4e58-aa35-367f13b91e17', '92de6bd1-222a-4fb5-bb41-120db189a0f7', 'd46b57f1-3a75-4338-8808-65582f8d94c8', '9f584829-b266-48fd-98cf-e86a4235fcbe', '17aab31e-3103-49c0-b8d6-9759bbe79a84', 'c14c1e16-eae8-4d5b-9ddf-b72d64c43125', '1492808e-c3c2-45e4-8518-cab2612e22e7', '66caa0a9-74a8-4cce-89cc-2312d253c6f7', '5a8460ef-028b-44c6-a4bd-d59dcb0796f7', 'ed20d25b-c10d-4f19-a527-ba3986779d48', '415de96b-e6bb-494e-b35e-966c7dd7a87f', '84fe4c94-9dd2-4322-9fbf-1016ee227291', '27fe07e1-d4f3-44bc-a492-2f9d1ddeb0c7', '302ea960-1230-4bf8-99fe-27b1f9067cc2', '894364b3-fb27-4540-a326-1f5d61b78745', '1684acf9-ef02-4e79-a090-bd3e2a427f05', '023631b0-dcd4-45ee-b15c-814f0cf38a3b', 'd6093c90-218a-4dfa-b851-e6683be1b50f', 'c7a0ff84-0f87-41e5-a8da-5257d8f22fbc', 'b6b65099-73c3-4abb-bd1c-842f26e18531', 'ba4e3148-fcf9-42fb-a0b8-8ab4b52ee982', '1a5ed077-8c88-47e4-ae7e-2008eb4c3e22', 'b6071397-8be8-4827-885a-45bbc6d1fb4c', '9feb48b1-9a04-4ba0-bc24-f63ecedfe23c', '3c6cb31b-9f2b-49df-8b06-50468e85fbf8', '31622995-c2ae-494f-b685-3bc348ebd21d', 'c08ae808-8493-4daa-ab35-daa380ee9df8', '80f8c2fa-c4c9-431c-9ae8-23007b27a85c', '34a888c2-3361-4ab5-a980-e3bfa6efc30d', '0904f757-7efb-44d1-a07c-24ff0833a5bf', '8c4347f0-f777-4bda-921c-83dedf80e1de', 'c92ee54a-0b8e-4a1d-abdc-ae1f5fc7826c', '96153093-3cfe-4742-904d-082fda5ae357', '08b7f452-154f-40dd-b211-38c0ab23a43b', '9c891958-ab84-4fde-a1ca-5d78eeaa394f', '1418516c-8077-40a6-9f51-41b9918afc9f', '024d6120-f86f-4e13-8155-9b20e55d99ad', 'c87fb18d-b128-44be-9e86-8ce394233d49', '34b1ff62-2372-473f-b93f-cca96e2eb61a', '56fe2f0a-c17c-4d2a-93ee-b06d2fbd3dff', 'd38d30d2-4da0-4138-97f9-211255fc288b', 'ab228ff2-1100-4100-ab63-88ea99a7e943', '506dd0b4-fee5-4dcc-874e-f0235bfa2828', '26685da3-b9c4-4ed8-a558-2ca25be64085', '7322b9d3-3a26-4da4-a750-af006850174b', '0514918f-5ffa-4e77-b220-8b64830a40a7', 'bf43b8cb-967c-4523-97f3-011be9762178', '7b5d0d66-16ad-4291-8b06-099a356cea4b', 'd5eddf04-7beb-4a20-a045-33d11424d8cc', '90a0a1ba-c8c2-4e00-ab60-42b4d59cc516', '4b414d7a-cbf0-4624-88de-c805138d9469', '6c173c55-d1c9-4285-9b10-e21d678e7999', 'a241e0cd-2994-4d43-93b2-100bf77f7da7', 'c6422c29-1ec6-4b3c-a7dc-d909e4b6c776', 'e17fa03f-00c2-4084-a0f3-87bf51645714', '5809b5c8-d3fb-4995-bbe9-46b35ba7a837', '501088ff-9114-47f3-a7e7-36457bd82fab', 'f936b0ec-fd29-495c-80b9-96023b5fa460', 'd08b5426-3ef3-46b2-ae28-910d7b7937ca', 'f01c43e6-1957-441b-898f-b2c6273ba646', '3b128da7-919a-4a50-a440-f685220695ab', '8818059e-66b1-4ac0-b638-d427ce75bfc4', 'dde81f60-5a15-44f3-b60c-c7b169e554a3', '16a1b6cc-dced-4ab1-b9be-0b664667a67f', '8819cc44-6594-44a9-a97a-6df141cf952d', '040e807f-1b40-4a81-a096-ea383cdec0ed', '56b7a25d-b800-4320-b4f2-aa1668ecf1da', 'd206b9af-4e0c-4a6f-899f-020da44952d7', '53cb9724-e752-4070-bb8e-da16f0be0ba8', '019aa88e-c52c-497c-a823-a89b306889b1', 'f68874ca-c5fd-498e-902c-d872a7d16ee8', '5d2a5b1f-1089-4250-8aaa-f525a55043d6', '04cbcd12-88b1-4650-a755-8ad8499df957', 'dfff59bb-507a-4c4e-b9fa-2cf08b448a44', 'b1995568-c79d-401a-ba73-390016f8a02b', 'a905c316-7ee3-41d2-9b55-6e12c3317b9a', '29a2382d-fddd-49d1-98ee-d0f129b4994f', 'c9b92b8a-b5d3-4d7a-9229-8a450f5ab16a', 'ce5c0cab-9e87-4057-b6df-4045faf73d59', 'af3b6cf3-10d3-4277-8d01-2f4a9d448120', '96dfd8dd-9c48-4fc9-b886-fcbc74b3a6aa', '5ec36386-f181-4891-a12b-6c37896b6bbd', '371b2cc3-9ef5-420f-923b-b6c32920bac0', '80b20778-c6c4-45d4-8710-e55b22534600', '17f91db9-6a80-49d6-be28-b95abf1b8dbe', '4a9b7c39-7855-4c17-80ab-6a3f09c2eff1', 'd835f022-ee2a-43af-999e-cfeccc326ad0', '9cbe8973-fcb1-48b2-944d-35e1183ccb08', 'e4ef434d-8c32-4783-8bfa-4b355dbede8c', '0a44e499-75e2-497c-9343-f1b4262c8ed5', '791c6bdd-fbd5-4b2d-9d9e-7c2c3d0be271', 'b6e46e39-96f2-4857-8b8c-05946f97398f', '68be3455-0d3d-4252-a25d-e98d1f93959c', 'db22c6ea-fbf0-4591-a740-fdb11f104cd6', '5436bb08-3024-4498-8189-60954bb6c170', '0750c8db-f160-4546-bc94-fb7a61cd5001', '6c07253d-cead-49a8-946f-7fe0844137bc', '54855400-8642-44e4-81cc-7075cfea6f37', 'ac26a1e1-5de8-40c3-84b2-3f604cd7e391', 'fae87739-ac61-4ea0-9923-f2bd1b7bcbb5', '2c0509a5-6ec8-4390-ae2d-02ba73bab59e', '2f006589-53f7-411d-8279-359c69fc2b1f', '6f788244-8f9b-46dd-b5e1-b19f3a5e6131', 'c24041ef-b239-40de-b347-df426e0a2073', 'c7e71994-63b8-4fe3-a6ef-f962925aa112', 'c2d5a1a4-760c-4554-9d08-9992f08d35af', 'f78ecdfd-cb09-4f8d-b5e7-6e1d6adf8f88', '1c2563c0-56d0-4848-b2c8-d85691baa0c2', '549c7520-0777-479c-9090-8c83d7ff44fe', '5b423e62-762a-4ab4-b45d-e3081843e829', '4c1cfa51-39f4-4782-8e18-e32bcdaaa296', 'e79a895e-c380-485d-ab80-5072fdd94906', '0cf1c80e-7b02-4407-b59d-9a87f051217b', '77cf495c-34d0-4bca-9533-835c7062484c', '50ba7074-a1f3-4c47-bdca-ebed33745690', '3776d9dd-4495-42e6-96f1-0900b1a651bc', 'd0351fa3-86b9-4e5f-b602-5cc26b2b4e1a', 'ba7a5c70-ef14-4223-940e-d9b65a5c2f44', '0eb15e18-7b3c-4baf-b6ba-49cce461f02d', 'ddea4258-9527-4c8e-aa23-04c9f39d454b', '74088f01-1e96-469c-94e0-f61dc139ab51', '0afd8c89-7ad5-49d8-a85e-39ed10b37b11', '9787f01c-8a0d-4df6-a587-5d30a565f15b', 'fe2952b9-e54c-4fcf-a3f4-0d1b0cdcf532', '128f391d-c159-49cf-a223-c29d146e605c', '2d422992-adc4-4918-8695-1be3ea54fad0', '5443a9e4-61b3-4a58-b622-e0865e818197', '95cb1ba4-475a-4fba-af27-6e299b8693d6', '78a13011-7ede-43d7-9926-d26a847a0367', 'a944dbaa-8f75-4eff-8e48-3600be37c913', '9ba225a5-a1cc-4ebd-bed6-c5b65c881038', 'e0444955-bb69-4d70-9137-acb2a70d80de', 'a1aff3ec-86a5-44d0-b936-91b5f6019078', '2ddf1d36-57da-4007-8272-96c8e3962564', '304386a2-c613-4f16-9327-3f6a2167058b', 'ea28d9ae-a068-4a1d-b763-7020fc76a49a', '9cb0355f-2e29-4d16-876c-0458a685a829', '52be54e4-79c9-469a-807b-0fb6d455b788', '64fc0b3b-d89c-47a4-baae-fc6d5cca946d', '43dfd9d5-6650-4815-8a95-9e55a0457528', 'd736675f-7033-400a-92c7-757bf805b112', '393cafae-c886-463a-8e0e-5138959e1a67', '51883c97-7a92-461f-a1e7-6d0f3ce88999', 'ef917ef5-ddbb-4398-828b-4467007c4b32', '030ac600-cb66-4641-8f22-af782a925ac0', '2a52bd60-903c-4fbe-97ab-4079853853b4', '847d3905-a68a-47cf-9569-f9ff452358dd', 'e631b00c-caa4-40a0-a619-1938e43365ed', '15ee5af7-6e29-4af8-ba31-65da47953658', 'c29771b5-48ee-4d68-9980-793abaf98b19', 'c10f7bd8-9d31-4d23-b0a9-f340ff3aa1d0', 'a49ae307-917e-4aa5-92ed-478f0bccc1d5', '74b6746c-d008-49ce-b752-583ff964da8c', '234a6ada-1f19-4af0-b30b-2715e9d6827c', '22a33e3b-2807-4070-b4c9-548afecb263a', '4d0530d8-d9dc-4311-b635-8903c20f89cb', 'f52103bb-f7e3-4164-a161-ef984538455a', 'bfbd37dc-5a95-4ffe-a366-16c9b9bf557d', '55ec0987-804b-499d-9d94-5fa8c1f94bb7', '204a52cf-b02c-43eb-a468-09c0d3a8bf5e', 'f840c68d-b8d4-411a-add9-79ddb4a5a01d', 'b0535bfe-3132-4a86-92f5-e737403415f1', '8be7bad4-d871-4e0b-b739-c83dba0a6b48', 'ee97aa03-f966-4850-b6fa-245ad3f06746', '52aa95e0-2ad7-461d-a359-3d6f0ba3bf00', '1ff650aa-2505-4f9f-afdc-62affd4a59f3', '491e29c6-4036-4672-9499-7b8d52381b75', '9139c65b-fffd-48de-9b9a-6f3e8ca3b58f', '3c66e192-a190-4d74-b725-52b496ae4e71', 'b397eafa-4896-45c8-88b7-296ab4c72f90', '4cb9b6e3-3826-4e84-8379-1f90b822e929', '90e9e729-d7f6-4641-ae0b-248da1d41c1b', '57f7700e-fa43-4490-afe8-06a649b1edef', 'eee3c5f8-5463-466a-8f2b-962983fda9d6', '6045b6c3-2ae7-499f-99a6-831538329df6', 'c9d6a978-b2f2-47ba-a226-5fba26d893dd', '89b2bca8-bf9e-4af0-9013-e7fdbd1ca74b', '52ee2513-9fcf-44a7-aec9-05cc0408de42', 'b21a3332-ba3c-4abd-ad81-710b1622e68e', '47e84f03-09ae-430d-9abb-d071de9c8f6f', '108d4049-e88b-475e-a248-afeaae2782ef', '5dc7ce26-be22-44be-b7f7-d7195e3d2a88', 'dda8d9e0-ecd1-43bd-aee7-9a9bd16ec428', '4bd1fd10-9ba1-436c-8752-27b7a4560ddb', '4095fe81-5265-4056-bf0d-9dc5f205f721', 'c19bee4f-1b7f-419f-a258-3024a1e24362', '2b1643a6-dfa2-4737-8167-5d2f44345f62', 'c6664015-b8d1-45c3-9e0f-30ed1fcbaace', 'a80b5b94-9818-4a0b-827e-ba7595f24f7d', '7350bb95-48a2-4e12-84e2-5adc4a27aeb1', 'ddc5f6f6-9c3c-4f88-8f6a-0226adcb5919', 'ba30fa04-e616-49f3-9d46-24e1d872b9a4', '3f9ff912-1e91-4605-9249-c40022e202d7', 'd2c30556-8bc6-4dd4-b2fe-c4c3c4ab5885', '6d3d901a-bdc5-47cd-8883-6b4effd4ff7c', 'e3932a7d-dfe7-44ae-bc26-c40afd5f96bf', 'a39488f8-1cda-4fef-92a3-0b397ee5f847', '696c2f1c-a5a2-4245-813e-3c11aa0c33bc', '4045f62d-ccb1-4d3d-bda4-1d85573774ae', 'ca6d2373-81b7-44ac-9383-0a7b44218b4a', '478c3c4a-8272-4ae3-b40c-3768035edbae', '40633562-4688-40d3-96ca-174c7e57eff2', 'ba1e7fed-7979-4496-9b2b-a296c1905013', 'f293b8f8-d4b6-42f2-bdbc-c596e6b8c9d0', '6e34871f-b153-4397-9f96-f967f99a340a', 'a4a9a24c-9bf6-4b4c-b27d-7928040a6be1', 'ab940563-ed69-49c3-9432-9dd0242c38dc', 'b704f1a5-a183-440d-9620-3a5eb025e872', '8760b3e3-dd7e-47c4-84f7-e6ffc9ed2e28', 'b1913a3c-939a-4179-8747-8894039bb9d2', 'b8bddd94-c115-48e8-836e-d2c0dd86b4c1', 'c81493e0-1680-4dc1-9ea9-342b2c6a2c3d', '120da6e4-646a-4e93-8c48-633ecb6b5463', '5b5148bf-3080-497e-bd0f-ee4e0f043ce7', '92fffecd-d047-4272-93f0-e34fe1095654', '1d12cf0b-ce2c-4a35-ba08-b3c207d5e833', 'b23e5e4f-2045-4010-a8cf-b08f78bb8a7f', '3576ff00-12fe-467d-b0b3-70c17e6fa7fb', 'c60b0ae9-3caa-4f4f-9fea-bc87ec18db01', 'fdd72e16-c00d-4f61-9064-fdab83067032', '7830c781-83c6-4a39-ab31-12e93e49971c', 'ef33333f-13b6-49b1-a93a-7e1814c1579c', '1dfe49de-acd2-4751-a309-010e4c80746e', 'd2667211-fce8-46e3-bb93-9fb6f96e2862', '2ec55b7b-20b7-4cf9-b1af-7eebc566c5fd', 'd29ed45d-0ac5-495e-b8f8-0a3f5bca1d68', 'a64c69a2-30d1-43d1-8168-276cdccf9e9a', '20067308-9771-4a1c-98cf-0897c25af809', '43d796a8-4572-4d85-9070-c0e2e2a12e83', 'fb3ef62d-77fc-4235-a38a-1cb6a8f5988b', '7a09516b-1eb1-497e-a6c2-2cbe33bcdaf1', 'a9dca19f-c4e5-423f-b3dd-42b299f03a05', '3775bd14-c4a6-451c-8b65-a80427853797', '100a2224-3dec-4029-b3b2-83910acd126d', '3878a9bc-f6d7-420e-b7cc-1c01a5b88a7d', '1f99ff9c-e148-4911-a841-72fc13a6715c', 'e7627f2d-71ea-47af-9726-506d71cb7008', 'b4a105c3-b28f-4ff4-bbf0-d077055c4047', 'b2064e5c-3e87-4d5b-bf26-f72f6a7f3057', '8bffddb8-9f0d-4702-97f8-260774038c37', 'f99db900-76dd-4e80-ad56-1b5ba62ed0b6', 'e4ceda65-fe1e-4e04-bc5b-095ef9d287ab', 'a06b53a5-5d79-453f-acee-6acc4233380d', 'b3bdb06f-cbba-45ca-b220-9ff15c0a70b3', '9e17b192-16d3-4e67-94d9-f60c52b202b2', 'd7216c28-bc23-4e22-8070-7ca66d10fbaa', '2abb3181-da0b-43ab-84b4-b05f9a71006c', '0783acfc-6542-4e0d-95dc-688beee0abaa', 'adb284a2-bad5-4764-b598-4ab9d292e393', '4e56ac00-e29a-425d-97dd-76a0eca4185c', '731a9eaf-948e-49c0-b104-291038854e3d', '1fc924be-e382-4bd0-b044-98cb6337cacf', 'd1051d18-d9e9-4cfb-9d34-f68b4f116570', '14ddd434-dce1-4c75-a4b6-ab9074d0eab2', '50ec2195-9903-46fa-b266-69167a128a5b', '200bbbcf-9a01-41b5-8b06-954e88759a42', 'ab8310f9-d92e-42e4-83d5-f04b8a826418', '6c9ebcca-bff1-479e-bb26-217cad961da4', '544d33c7-2d7b-4ceb-8cc6-bbfacada9ba9', 'df0a6bfc-a117-4ec8-b2b8-7da7eccd33d8', '31086c8b-ca13-444e-b1aa-e37ecde17fab', '82fccaa9-2976-405b-bc44-444cff9c2815', '6dff6735-c2c5-46b0-8a41-69167bf2fdf7', '5dcdcebb-f642-4dce-a91e-c5a5cc7295bc', '34c6808c-3d1b-4940-bad9-194e18819726', '914b7b3d-7c0f-4f3b-a1d9-38f22996809c', 'b4d9941c-dbf9-4f20-a7fe-bc18cdd88f46', 'ec479475-c53c-4241-8214-59f5cbb0cba0', '2e52f6c4-8141-45a1-96bc-b013a4f4250a', '3ea47f07-37cd-4213-8b55-0987189f3fb5', '98f8dbec-5823-4e7b-b1da-4a0a66dd1841', '6ac1b426-5073-4f20-a6d7-1223585831f1', '3a68d0db-c832-4dfa-9990-900d2ae4052e', 'dc24ea82-98be-49f1-abd8-30a387a82c69', '7b2d625e-8cc2-42fb-9e29-b34aa426c58c', '505644f1-8bcf-443c-910e-56529d51d2ea', '615c020d-21f5-4546-a5ee-fd44d03d85fc', '1d58f4e4-9bea-4bdd-b2c8-cd8f0d9c453b', '63513359-8a62-411b-befc-832b186e26cd', '7d06e6b0-c3f6-4ae1-8ca9-b8caa6855a5d', '85cdf949-7be1-4b5f-92e1-e3727eb8fe94', '387d0ce1-8293-4dce-9768-a1c209033753', 'a0ad096f-db7f-4482-8cc2-7351d09f0e73', 'fce2e2ed-a8d1-4aec-adcf-7994c0652e5b', '3f6b4f1f-7647-4366-a686-c9e305a630c2', '30851f37-ca48-46ec-9eef-a58f074d08ef', '8a0c460a-3834-4fc6-a8d6-e9c8543d36ab', '28b73b0f-d5b1-4281-8525-9fb24f4b217c', '0aea1091-b42a-4c8a-a204-27c189cc5ed8', '6bd99aa1-0b1d-415d-af9c-94f28f1a0042', '919e9c5d-58f7-4390-a195-3a9f9a5171ed', '42600244-505c-4e82-be7b-426daa23b7fd', '9fc4c493-901a-4947-860e-43db9656684b', 'f61d1915-5424-4fb8-9846-ad37acfd3909', '1f2318ea-b550-4abf-b342-a78c067ce3af', '4695169d-6637-40db-b9c3-a14133da2bad', '55f38b46-4ad2-41c6-9723-a684ad69e88d', '456a3a88-691e-4539-9ecf-0f80d149e8c8', '43dec340-cc37-4cf6-8ed3-e1d0f1c61fc4', 'dbcbb0d3-532c-4074-9392-4551fc760b23', 'a04a033f-037a-434a-bd51-e902c27d7cf4'} {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  471.538938] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] Fetching Policy NSX-T metadata for Type:Segment QoS.
[  471.555258] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock Segment QoS taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 122 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  471.555440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  471.555569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  471.555994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  471.556173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  471.556345] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] The number of outdated resources for Type:Segment QoS Is:0.
[  471.556475] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Outdated resources of Type:Segment QoS Are:set() {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  471.556607] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Members).
[  473.011183] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  473.011615] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  474.293102] env[60890]: DEBUG networking_nsxv3.common.locking [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/bin/neutron-nsxv3-agent, line 10 in <module>>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 331 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 151 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 53 in __init__>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 505 in _dryrun>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 125 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  474.293892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  474.294182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  474.294662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  474.330721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  474.335342] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] [Policy] The number of outdated resources for Type:Security Group (Members) Is:0.
[  474.335651] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Outdated resources of Type:Security Group (Members) Are:set() {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  474.336060] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Inventory metadata have been refreshed.
[  474.337190] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Dryrun:True. Metadata refresh completed.
[  474.337687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  474.344164] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.agent [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] VMware NSXv3 Agent initialized successfully.
[  474.346463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquiring lock "singleton_lock" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  474.347156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Acquired lock "singleton_lock" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  474.347890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Releasing lock "singleton_lock" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  474.348669] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Full set of CONF: {{(pid=60890) _wait_for_exit_or_signal /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/service.py:363}}
[  474.348929] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ******************************************************************************** {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2600}}
[  474.349178] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] Configuration options gathered from: {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2601}}
[  474.349416] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] command line args: ['--config-file', '/etc/neutron/neutron.conf', '--config-file', '/etc/neutron/plugins/ml2/ml2_conf.ini', '--config-file', '/etc/neutron/plugins/ml2/ml2-nsxv3-1.ini'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2602}}
[  474.349711] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] config files: ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini', '/etc/neutron/plugins/ml2/ml2-nsxv3-1.ini'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2603}}
[  474.349964] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ================================================================================ {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2605}}
[  474.350322] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] agent_down_time                = 75 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.350623] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] allow_automatic_dhcp_failover  = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.350898] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] allow_bulk                     = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.351178] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] api_extensions_path            =  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.351455] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] api_paste_config               = api-paste.ini {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.351732] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] auth_strategy                  = keystone {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.352055] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] backdoor_port                  = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.352288] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] backdoor_socket                = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.352600] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] backlog                        = 4096 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.352881] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] base_mac                       = fa:16:3e:00:00:00 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.353187] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] bind_host                      = 0.0.0.0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.353466] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] bind_port                      = 9696 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.353738] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] client_socket_timeout          = 900 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.354014] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] config_dir                     = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.354260] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] config_file                    = ['/etc/neutron/neutron.conf', '/etc/neutron/plugins/ml2/ml2_conf.ini', '/etc/neutron/plugins/ml2/ml2-nsxv3-1.ini'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.354544] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] config_source                  = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.354850] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] control_exchange               = neutron {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.355145] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] core_plugin                    = ml2 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.355428] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] debug                          = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.355721] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] default_availability_zones     = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.356108] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'OFPHandler=INFO', 'OfctlService=INFO', 'os_ken.base.app_manager=INFO', 'os_ken.controller.controller=INFO'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.356438] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] dhcp_agent_notification        = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.356732] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] dhcp_agents_per_network        = 1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.357017] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] dhcp_lease_duration            = 86400 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.357300] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] dhcp_load_type                 = networks {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.357588] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] dns_domain                     = openstacklocal {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.357870] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] dvr_base_mac                   = fa:16:3f:00:00:00 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.358143] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] enable_default_route_bfd       = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.358411] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] enable_default_route_ecmp      = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.358678] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] enable_dvr                     = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.358950] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] enable_new_agents              = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.359293] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] enable_services_on_agents_with_admin_state_down = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.359577] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] enable_signals                 = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.359851] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] enable_traditional_dhcp        = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.360155] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] external_dns_driver            = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.360426] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] filter_validation              = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.360720] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] global_physnet_mtu             = 8950 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.360995] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] graceful_shutdown_timeout      = 60 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.361318] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] host                           = cpu-1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.361603] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] host_dvr_for_dhcp              = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.361876] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] http_retries                   = 3 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.362163] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] instance_format                = [instance: %(uuid)s]  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.362455] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.362735] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ipam_driver                    = internal {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.363010] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ipv6_pd_enabled                = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.363276] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] l3_ha                          = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.363562] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] l3_ha_net_cidr                 = 169.254.192.0/18 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.363838] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] l3_ha_network_physical_name    =  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.364126] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] l3_ha_network_type             =  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.364416] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] log_config_append              = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.364726] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.365009] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] log_dir                        = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.365289] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] log_file                       = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.365532] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] log_options                    = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.365849] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] log_rotate_interval            = 1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.366151] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] log_rotate_interval_type       = days {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.366434] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] log_rotation_type              = none {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.366684] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.366933] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.367213] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.367491] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.367723] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.368040] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] max_dns_nameservers            = 5 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.368317] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] max_header_line                = 16384 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.368587] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] max_l3_agents_per_router       = 3 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.368869] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] max_logfile_count              = 30 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.369133] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] max_logfile_size_mb            = 200 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.369408] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] max_subnet_host_routes         = 20 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.369683] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] my_ip                          = 10.180.1.21 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.369968] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] my_ipv6                        = ::1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.370246] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] network_auto_schedule          = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.370519] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] network_link_prefix            = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.370808] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] network_scheduler_driver       = neutron.scheduler.dhcp_agent_scheduler.WeightScheduler {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.371283] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] notify_nova_on_port_data_changes = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.371569] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] notify_nova_on_port_status_changes = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.371862] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] pagination_max_limit           = -1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.372149] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] publish_errors                 = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.372445] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] rate_limit_burst               = 0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.372755] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] rate_limit_except_level        = CRITICAL {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.373036] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] rate_limit_interval            = 0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.373306] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] retry_until_window             = 30 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.373575] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] router_distributed             = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.373844] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] rpc_resources_processing_step  = 20 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.374123] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] rpc_response_max_timeout       = 600 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.374391] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] rpc_response_timeout           = 60 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.374682] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] send_events_interval           = 2 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.374979] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] service_plugins                = ['router'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.375282] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] setproctitle                   = on {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.375575] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] state_path                     = /opt/stack/data/neutron {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.375857] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] syslog_log_facility            = LOG_USER {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.376163] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] tcp_keepidle                   = 600 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.376442] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] transport_url                  = **** {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.376716] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] use_eventlog                   = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.377008] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] use_journal                    = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.377280] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] use_json                       = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.377549] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] use_ssl                        = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.377816] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] use_stderr                     = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.378090] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] use_syslog                     = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.378357] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] vlan_transparent               = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.378635] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] watch_log_file                 = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.378913] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] wsgi_default_pool_size         = 100 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.379186] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] wsgi_keep_alive                = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.379460] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] wsgi_log_format                = %(client_ip)s "%(request_line)s" status: %(status_code)s  len: %(body_length)s time: %(wall_seconds).7f {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.379741] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] wsgi_server_debug              = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2613}}
[  474.380058] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_concurrency.disable_process_locking = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.380358] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_concurrency.lock_path     = /opt/stack/data/neutron/lock {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.380642] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.380922] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.381201] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.381491] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.381766] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.382055] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.connection_string     = messaging:// {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.382343] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.enabled               = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.382630] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.es_doc_type           = notification {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.382926] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.es_scroll_size        = 10000 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.383209] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.es_scroll_time        = 2m {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.383481] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.filter_error_trace    = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.383759] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.hmac_keys             = **** {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.384075] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.sentinel_service_name = mymaster {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.384377] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.socket_timeout        = 0.1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.384672] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.trace_requests        = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.384958] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler.trace_sqlalchemy      = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.385244] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler_jaeger.process_tags   = {} {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.385512] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler_jaeger.service_name_prefix = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.385804] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] profiler_otlp.service_name_prefix = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.386109] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.enforce_new_defaults = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.386400] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.enforce_scope      = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.386703] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.policy_default_rule = default {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.387012] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.387300] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.policy_file        = /etc/neutron/policy.json {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.387599] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.387903] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.388212] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.388484] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.388760] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.389067] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.389359] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] agent.extensions               = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.389652] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.389934] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] service_providers.service_provider = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.390231] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.default_quota           = -1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.390528] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_driver            = neutron.db.quota.driver_nolock.DbQuotaNoLockDriver {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.390808] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_floatingip        = 50 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.391086] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_network           = 100 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.391359] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_port              = 500 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.391630] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_rbac_policy       = 10 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.391904] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_router            = 10 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.392202] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_security_group    = 10 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.392480] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_security_group_rule = 100 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.392777] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.quota_subnet            = 100 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.393063] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] QUOTAS.track_quota_usage       = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.393350] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep.capabilities           = [21, 12, 1, 2, 19] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.393638] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep.group                  = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.393910] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep.helper_command         = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.394185] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep.logger_name            = oslo_privsep.daemon {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.394459] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep.thread_pool_size       = 8 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.394749] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep.user                   = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.395046] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_dhcp_release.capabilities = [21, 12] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.395324] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_dhcp_release.group     = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.395591] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_dhcp_release.helper_command = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.395925] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_dhcp_release.logger_name = oslo_privsep.daemon {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.396283] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_dhcp_release.thread_pool_size = 8 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.396576] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_dhcp_release.user      = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.396868] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_ovs_vsctl.capabilities = [21, 12] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.397237] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_ovs_vsctl.group        = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.397520] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_ovs_vsctl.helper_command = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.397798] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_ovs_vsctl.logger_name  = oslo_privsep.daemon {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.398075] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_ovs_vsctl.thread_pool_size = 8 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.398356] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_ovs_vsctl.user         = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.398642] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_namespace.capabilities = [21] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.398913] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_namespace.group        = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.399193] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_namespace.helper_command = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.399464] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_namespace.logger_name  = oslo_privsep.daemon {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.399751] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_namespace.thread_pool_size = 8 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.400052] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_namespace.user         = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.400334] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_conntrack.capabilities = [12] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.400602] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_conntrack.group        = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.400873] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_conntrack.helper_command = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.401163] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_conntrack.logger_name  = oslo_privsep.daemon {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.401440] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_conntrack.thread_pool_size = 8 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.401722] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_conntrack.user         = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.402001] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_link.capabilities      = [12, 21] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.402268] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_link.group             = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.402533] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_link.helper_command    = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.402846] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_link.logger_name       = oslo_privsep.daemon {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.403127] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_link.thread_pool_size  = 8 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.403411] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] privsep_link.user              = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.403712] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.agent_id                 = nsxm-l-01a.corp.local {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.404019] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.agent_prometheus_exporter_port = 8000 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.404331] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.continue_on_failed_promotions = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.404504] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.dscp                     = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.404708] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.dscp_inherit             = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.404875] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.force_mp_to_policy       = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.405028] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.locking_coordinator_url  = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.405177] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.log_agent_heartbeats     = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.405326] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.logging_expire           = 1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.405482] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.logging_url              = unix:///var/run/redis/socket/redis.sock {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.405655] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.max_sg_tags_per_segment_port = 27 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.405849] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.polling_interval         = 240 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.406009] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.ports_tag_number_decrease_workaround = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.406165] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.quitting_rpc_timeout     = 120 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.406322] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.report_interval          = 30.0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.406478] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.root_helper              = sudo /opt/stack/data/venv/bin/neutron-rootwrap /etc/neutron/rootwrap.conf {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.406629] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.root_helper_daemon       = sudo /opt/stack/data/venv/bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.406783] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.rpc_max_records_per_query = 1000 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.406930] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.sync_skew                = 60 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.407076] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.synchronization_queue_size = 20 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.407222] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT.use_helper_for_ns_read   = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.407374] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] SECURITYGROUP.enable_ipset     = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.407526] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] SECURITYGROUP.enable_security_group = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.407677] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] SECURITYGROUP.firewall_driver  = nsxv3 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.407828] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] SECURITYGROUP.permitted_ethertypes = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.407980] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] experimental.ipv6_pd_enabled   = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.408149] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] experimental.linuxbridge       = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.408315] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.extension_drivers          = ['port_security'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.408462] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.external_network_type      = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.408621] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.mechanism_drivers          = ['nsxv3', 'openvswitch', 'logger'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.408776] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.overlay_ip_version         = 4 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.408926] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.path_mtu                   = 8950 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.409072] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.physical_network_mtus      = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.409230] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.tenant_network_types       = ['vlan'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.409381] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.tunnelled_network_rp_name  = rp_tunnelled {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.409534] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ml2.type_drivers               = ['vlan'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.409688] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT_CLI.neutron_port_id      = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.409836] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT_CLI.neutron_qos_policy_id = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.409980] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] AGENT_CLI.neutron_security_group_id = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.410134] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.mp_to_policy_retry_count = 10 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.410277] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.mp_to_policy_retry_sleep = 2 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.410421] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_concurrent_requests = 40 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.410572] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_connection_retry_count = 2 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.410720] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_connection_retry_sleep = 2 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.410872] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_default_policy_infrastructure_rules = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.411025] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_dfw_connectivity_strategy = NONE {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.411175] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_ip_discovery_switching_profile = nsx-default-ip-discovery-vlan-profile {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.411372] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_login_hostname     = nsxt.osci.c.eu-de-1.cloud.sap {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.411524] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_login_password     = **** {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.411759] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_login_port         = 443 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.411939] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_login_user         = admin {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.412111] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_managed_hosts      = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.412267] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_max_records_per_query = 200 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.412415] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_operation_retry_count = 3 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.412592] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_operation_retry_sleep = 5 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.412733] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_realization_timeout = 300 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.412891] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_remove_orphan_ports_after = 2.0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.413043] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_request_timeout    = 60 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.413191] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_requests_per_second = 80 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.413337] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_requests_per_second_timeout = 7 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.413489] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_spoof_guard_switching_profile = nsx-default-spoof-guard-vif-profile {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.413644] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_suppress_ssl_warnings = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.413795] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] NSXV3.nsxv3_transport_zone_name = nsx-vlan-transportzone {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.413991] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] vsphere.vsphere_login_hostname = vc-l-01a.corp.local {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.414145] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] vsphere.vsphere_login_password = VMware1! {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.414295] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] vsphere.vsphere_login_username = administrator@vsphere.local {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.414439] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] vsphere.vsphere_suppress_ssl_wornings = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.414590] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.auth_section              = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.414776] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.auth_type                 = password {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.414934] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.cafile                    = /opt/stack/data/ca-bundle.pem {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.415077] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.certfile                  = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.415222] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.collect_timing            = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.415373] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.endpoint_type             = public {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.415517] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.insecure                  = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.415664] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.keyfile                   = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.415834] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.region_name               = RegionOne {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.415987] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.split_loggers             = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.416156] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] nova.timeout                   = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.416309] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.auth_section         = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.416464] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.auth_type            = password {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.416615] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.cafile               = /opt/stack/data/ca-bundle.pem {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.416760] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.certfile             = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.416904] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.collect_timing       = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.417080] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.endpoint_type        = public {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.417227] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.insecure             = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.417365] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.keyfile              = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.417501] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.region_name          = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.417674] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.split_loggers        = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.417783] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] placement.timeout              = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.417929] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.auth_section            = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.418070] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.auth_type               = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.418206] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.cafile                  = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.418342] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.certfile                = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.418482] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.collect_timing          = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.418617] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.connect_retries         = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.418758] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.connect_retry_delay     = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.418900] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.enable_notifications    = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.419038] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.endpoint_override       = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.419180] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.insecure                = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.419318] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.interface               = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.419455] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.keyfile                 = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.419593] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.max_version             = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.419730] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.min_version             = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.419868] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.region_name             = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.420044] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.retriable_status_codes  = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.420159] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.service_name            = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.420295] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.service_type            = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.420435] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.split_loggers           = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.420571] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.status_code_retries     = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.420710] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.status_code_retry_delay = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.420850] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.timeout                 = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.420989] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.valid_interfaces        = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.421124] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ironic.version                 = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.421268] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] cli_script.dry_run             = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.421425] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.421567] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.421711] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.421948] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.422103] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.422248] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.422392] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.heartbeat_in_pthread = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.422537] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.heartbeat_rate = 3 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.422707] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.422836] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.hostname = devstack {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.422975] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.423126] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.423276] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.423427] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.423577] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.processname = neutron-nsxv3-agent {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.423726] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.423871] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.424037] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.424186] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.424328] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_quorum_delivery_limit = 0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.424472] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_bytes = 0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.424653] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_quorum_max_memory_length = 0 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.424779] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.424931] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.425075] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.425217] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_stream_fanout = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.425361] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.425502] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rabbit_transient_quorum_queue = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.425658] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.425805] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.ssl      = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.425960] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.426112] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.426254] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.426406] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.426556] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.ssl_version =  {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.426699] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_rabbit.use_queue_manager = False {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.426856] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_notifications.driver = [] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.427017] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_notifications.retry = -1 {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.427188] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.427334] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] oslo_messaging_notifications.transport_url = **** {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2620}}
[  474.427445] env[60890]: DEBUG oslo_service.service [None req-00c3e928-a011-49ac-a50e-ec83956f086a None None] ******************************************************************************** {{(pid=60890) log_opt_values /opt/stack/data/venv/lib/python3.10/site-packages/oslo_config/cfg.py:2624}}
[  474.428464] env[60890]: DEBUG neutron.agent.securitygroups_rpc [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] Init firewall settings (driver=nsxv3) {{(pid=60890) init_firewall /opt/stack/neutron/neutron/agent/securitygroups_rpc.py:124}}
[  474.429768] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.extensions.firewall [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] Initializing NSXv3SecurityGroupsDriver dummy driver. {{(pid=60890) __init__ /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/extensions/firewall.py:16}}
[  474.429996] env[60890]: INFO neutron.plugins.ml2.drivers.agent._common_agent [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] RPC agent_id: nsxm-l-01a.corp.local
[  474.431379] env[60890]: INFO neutron.agent.agent_extensions_manager [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] Loaded agent extensions: []
[  474.431832] env[60890]: DEBUG neutron_lib.callbacks.manager [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] Publish callbacks [] for NSXv3 Agent (None), after_init {{(pid=60890) _notify_loop /opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/callbacks/manager.py:184}}
[  474.433997] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 35e950ee958e482b9cb70c7a43108064 in queue reply_156db6c4256a4c58884afce15e6f5340
[  474.445926] env[60890]: INFO oslo.messaging._drivers.impl_rabbit [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] Creating fanout queue: q-agent-notifier-port-update_fanout_35fd2832d2f44d4885d748c1aab29683
[  474.450237] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 35e950ee958e482b9cb70c7a43108064
[  474.457521] env[60890]: INFO oslo.messaging._drivers.impl_rabbit [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] Creating fanout queue: q-agent-notifier-port-delete_fanout_843fb06353f94f81afa25a13e52418e2
[  474.468262] env[60890]: INFO oslo.messaging._drivers.impl_rabbit [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] Creating fanout queue: q-agent-notifier-security_group-update_fanout_9fe5a91feb274170965de4e63e8b2f2c
[  474.601131] env[60890]: INFO oslo.messaging._drivers.impl_rabbit [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] Creating fanout queue: q-agent-notifier-nsxv3-update_fanout_6a8dc3b7649a42c6b221a667f314674a
[  474.604681] env[60890]: INFO neutron.plugins.ml2.drivers.agent._common_agent [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] NSXv3 Agent Agent RPC Daemon Started!
[  474.604944] env[60890]: INFO neutron.plugins.ml2.drivers.agent._common_agent [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] NSXv3 Agent Agent loop - iteration:0 started
[  474.605161] env[60890]: INFO neutron.plugins.ml2.drivers.agent._common_agent [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] NSXv3 Agent Agent out of sync with plugin!
[  478.012354] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  478.012774] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  478.280177] env[60890]: INFO networking_nsxv3.common.synchronization [None req-0e630818-89f8-40d8-9773-93729ea55567 demo admin] Enqueued Resource ID: 718c0ad6-d83f-4caf-b4d6-52cc2466c7c9 with Priority: HIGHEST for action security_group_rules
[  478.280692] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 718c0ad6-d83f-4caf-b4d6-52cc2466c7c9 with Priority: HIGHEST for action security_group_rules
[  478.281188] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  478.281305] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  478.281339] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  478.281627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  478.281723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  478.281902] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-718c0ad6-d83f-4caf-b4d6-52cc2466c7c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  478.282004] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-718c0ad6-d83f-4caf-b4d6-52cc2466c7c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  478.282064] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-718c0ad6-d83f-4caf-b4d6-52cc2466c7c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  478.282269] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-718c0ad6-d83f-4caf-b4d6-52cc2466c7c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  478.282400] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('718c0ad6-d83f-4caf-b4d6-52cc2466c7c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  478.282664] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9f5f7175af434a0db49fce8d6a9115ae in queue reply_156db6c4256a4c58884afce15e6f5340
[  478.295774] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9f5f7175af434a0db49fce8d6a9115ae
[  478.802750] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  478.803012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  478.803012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  478.803276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  478.803351] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  478.803392] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 718c0ad6-d83f-4caf-b4d6-52cc2466c7c9 already deleted.
[  478.803467] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-718c0ad6-d83f-4caf-b4d6-52cc2466c7c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  483.282138] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  483.282540] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  488.284050] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  488.284387] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  493.285014] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  493.285354] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  494.354140] env[60890]: INFO networking_nsxv3.common.synchronization [None req-c0897f99-73ab-408c-8eb3-cfb53b630661 None None] Enqueued Resource ID: 9918b866-181d-43ce-885f-12854632beee with Priority: HIGHEST for action port
[  494.354762] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9918b866-181d-43ce-885f-12854632beee with Priority: HIGHEST for action port
[  494.355048] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  494.355177] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  494.355326] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  494.355442] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  494.355529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  494.355761] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-9918b866-181d-43ce-885f-12854632beee taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  494.355868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-9918b866-181d-43ce-885f-12854632beee" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  494.355927] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-9918b866-181d-43ce-885f-12854632beee" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  494.356159] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-9918b866-181d-43ce-885f-12854632beee" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  494.356238] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9918b866-181d-43ce-885f-12854632beee',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  494.356440] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 489203d42c5c47949e237079e79c3b78 in queue reply_156db6c4256a4c58884afce15e6f5340
[  494.619626] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 489203d42c5c47949e237079e79c3b78
[  494.858978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-9918b866-181d-43ce-885f-12854632beee" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  494.859872] env[60890]: Traceback (most recent call last):
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  494.859956] env[60890]:     timer()
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  494.859956] env[60890]:     cb(*args, **kw)
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  494.859956] env[60890]:     result = function(*args, **kwargs)
[  494.859956] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  494.859956] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  494.859956] env[60890]:     return method(*args, **kwargs)
[  494.859956] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  494.859956] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  494.859956] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  494.859956] env[60890]:     result = self.transport._send(
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  494.859956] env[60890]:     return self._driver.send(target, ctxt, message,
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  494.859956] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  494.859956] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  494.859956] env[60890]:     raise result
[  494.859956] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  494.859956] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  496.438265] env[60890]: INFO networking_nsxv3.common.synchronization [None req-17823e50-2be7-4f98-b9bf-fad1ff0fa923 None None] Enqueued Resource ID: 06f296e0-a5b6-4ef7-a68e-6b92914d40c3 with Priority: HIGHEST for action port
[  496.438521] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 06f296e0-a5b6-4ef7-a68e-6b92914d40c3 with Priority: HIGHEST for action port
[  496.439098] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  496.439225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  496.439293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  496.439489] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  496.439571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  496.439717] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-06f296e0-a5b6-4ef7-a68e-6b92914d40c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  496.439813] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-06f296e0-a5b6-4ef7-a68e-6b92914d40c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  496.439875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-06f296e0-a5b6-4ef7-a68e-6b92914d40c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  496.440091] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-06f296e0-a5b6-4ef7-a68e-6b92914d40c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  496.440171] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('06f296e0-a5b6-4ef7-a68e-6b92914d40c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  496.440346] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0c5ee1b166814b0a8106ab6da8729faf in queue reply_156db6c4256a4c58884afce15e6f5340
[  496.448509] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0c5ee1b166814b0a8106ab6da8729faf
[  496.945262] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-06f296e0-a5b6-4ef7-a68e-6b92914d40c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  496.945262] env[60890]: Traceback (most recent call last):
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  496.945262] env[60890]:     timer()
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  496.945262] env[60890]:     cb(*args, **kw)
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  496.945262] env[60890]:     result = function(*args, **kwargs)
[  496.945262] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  496.945262] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  496.945262] env[60890]:     return method(*args, **kwargs)
[  496.945262] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  496.945262] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  496.945262] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  496.945262] env[60890]:     result = self.transport._send(
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  496.945262] env[60890]:     return self._driver.send(target, ctxt, message,
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  496.945262] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  496.945262] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  496.945262] env[60890]:     raise result
[  496.945262] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  496.945262] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  499.088054] env[60890]: INFO networking_nsxv3.common.synchronization [None req-b7e86256-425c-4dbd-a323-0f86e816f095 None None] Enqueued Resource ID: ae1bec6c-fbc2-4a60-88de-b5cb828e16dc with Priority: HIGHEST for action port
[  499.088335] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ae1bec6c-fbc2-4a60-88de-b5cb828e16dc with Priority: HIGHEST for action port
[  499.088864] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  499.088985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  499.089051] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  499.089243] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  499.089325] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  499.089471] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-ae1bec6c-fbc2-4a60-88de-b5cb828e16dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  499.089588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-ae1bec6c-fbc2-4a60-88de-b5cb828e16dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  499.089620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-ae1bec6c-fbc2-4a60-88de-b5cb828e16dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  499.089799] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-ae1bec6c-fbc2-4a60-88de-b5cb828e16dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  499.089895] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1bec6c-fbc2-4a60-88de-b5cb828e16dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  499.090059] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a0e787df2c514e3f92c4e604deef626f in queue reply_156db6c4256a4c58884afce15e6f5340
[  499.222943] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a0e787df2c514e3f92c4e604deef626f
[  499.595317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-ae1bec6c-fbc2-4a60-88de-b5cb828e16dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  499.595560] env[60890]: Traceback (most recent call last):
[  499.595610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  499.595610] env[60890]:     timer()
[  499.595610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  499.595610] env[60890]:     cb(*args, **kw)
[  499.595610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  499.595610] env[60890]:     result = function(*args, **kwargs)
[  499.595610] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  499.595610] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  499.595610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  499.595610] env[60890]:     return method(*args, **kwargs)
[  499.595610] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  499.595610] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  499.595610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  499.595610] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  499.595610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  499.595610] env[60890]:     result = self.transport._send(
[  499.595610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  499.595610] env[60890]:     return self._driver.send(target, ctxt, message,
[  499.596052] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  499.596052] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  499.596052] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  499.596052] env[60890]:     raise result
[  499.596052] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  499.596052] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  501.029882] env[60890]: INFO networking_nsxv3.common.synchronization [None req-08755c6b-1a78-4d24-8340-f7c587d5ec6f None None] Enqueued Resource ID: e13a1619-8215-4b39-8655-eff41734fe73 with Priority: HIGHEST for action port
[  501.030159] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e13a1619-8215-4b39-8655-eff41734fe73 with Priority: HIGHEST for action port
[  501.030681] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  501.030815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  501.030925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  501.031079] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  501.031172] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  501.031316] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-e13a1619-8215-4b39-8655-eff41734fe73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  501.031421] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-e13a1619-8215-4b39-8655-eff41734fe73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  501.031469] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-e13a1619-8215-4b39-8655-eff41734fe73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  501.031666] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-e13a1619-8215-4b39-8655-eff41734fe73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  501.031738] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e13a1619-8215-4b39-8655-eff41734fe73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  501.031919] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 012ef3706e1a408a946ac20d725a692d in queue reply_156db6c4256a4c58884afce15e6f5340
[  501.059317] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 012ef3706e1a408a946ac20d725a692d
[  501.533766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-e13a1619-8215-4b39-8655-eff41734fe73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  501.537049] env[60890]: Traceback (most recent call last):
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  501.537049] env[60890]:     timer()
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  501.537049] env[60890]:     cb(*args, **kw)
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  501.537049] env[60890]:     result = function(*args, **kwargs)
[  501.537049] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  501.537049] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  501.537049] env[60890]:     return method(*args, **kwargs)
[  501.537049] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  501.537049] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  501.537049] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  501.537049] env[60890]:     result = self.transport._send(
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  501.537049] env[60890]:     return self._driver.send(target, ctxt, message,
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  501.537049] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  501.537049] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  501.537049] env[60890]:     raise result
[  501.537049] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  501.537049] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  503.225964] env[60890]: INFO networking_nsxv3.common.synchronization [None req-32de0408-8f7e-4f0a-a88e-c8d3ef1d84f8 None None] Enqueued Resource ID: 830a5dce-9375-455f-866a-faa616e452ab with Priority: HIGHEST for action port
[  503.227840] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 830a5dce-9375-455f-866a-faa616e452ab with Priority: HIGHEST for action port
[  503.227840] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  503.227840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  503.227840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  503.227840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  503.227840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  503.227840] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-830a5dce-9375-455f-866a-faa616e452ab taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  503.227840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-830a5dce-9375-455f-866a-faa616e452ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  503.227840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-830a5dce-9375-455f-866a-faa616e452ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  503.227840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-830a5dce-9375-455f-866a-faa616e452ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  503.227840] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('830a5dce-9375-455f-866a-faa616e452ab',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  503.228362] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 71d69a5cc29a49c79492ededc95c05a5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  503.332106] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 71d69a5cc29a49c79492ededc95c05a5
[  503.666871] env[60890]: INFO networking_nsxv3.common.synchronization [None req-723eebc1-e622-4bd1-9236-8e1b5dee3b4e None None] Enqueued Resource ID: 1d16238e-02a8-43b3-b72e-d8676f2eb174 with Priority: HIGHEST for action port
[  503.667409] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 1d16238e-02a8-43b3-b72e-d8676f2eb174 with Priority: HIGHEST for action port
[  503.668317] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  503.668317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  503.668317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  503.668317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  503.668317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  503.668645] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-1d16238e-02a8-43b3-b72e-d8676f2eb174 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  503.668645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-1d16238e-02a8-43b3-b72e-d8676f2eb174" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  503.668645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-1d16238e-02a8-43b3-b72e-d8676f2eb174" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  503.668823] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-1d16238e-02a8-43b3-b72e-d8676f2eb174" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  503.668878] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d16238e-02a8-43b3-b72e-d8676f2eb174',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  503.669059] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ce156e62dbfb4c2f85f7c909951af072 in queue reply_156db6c4256a4c58884afce15e6f5340
[  503.730152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-830a5dce-9375-455f-866a-faa616e452ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  503.730389] env[60890]: Traceback (most recent call last):
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  503.730448] env[60890]:     timer()
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  503.730448] env[60890]:     cb(*args, **kw)
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  503.730448] env[60890]:     result = function(*args, **kwargs)
[  503.730448] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  503.730448] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  503.730448] env[60890]:     return method(*args, **kwargs)
[  503.730448] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  503.730448] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  503.730448] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  503.730448] env[60890]:     result = self.transport._send(
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  503.730448] env[60890]:     return self._driver.send(target, ctxt, message,
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  503.730448] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  503.730448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  503.730448] env[60890]:     raise result
[  503.730448] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  503.730448] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  503.781231] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ce156e62dbfb4c2f85f7c909951af072
[  504.175139] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-1d16238e-02a8-43b3-b72e-d8676f2eb174" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  504.175375] env[60890]: Traceback (most recent call last):
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  504.175443] env[60890]:     timer()
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  504.175443] env[60890]:     cb(*args, **kw)
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  504.175443] env[60890]:     result = function(*args, **kwargs)
[  504.175443] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  504.175443] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  504.175443] env[60890]:     return method(*args, **kwargs)
[  504.175443] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  504.175443] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  504.175443] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  504.175443] env[60890]:     result = self.transport._send(
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  504.175443] env[60890]:     return self._driver.send(target, ctxt, message,
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  504.175443] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  504.175443] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  504.175443] env[60890]:     raise result
[  504.175443] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  504.175443] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  504.280050] env[60890]: INFO networking_nsxv3.common.synchronization [None req-205859dc-bdd5-473e-973b-bf3849c672e6 None None] Enqueued Resource ID: 413013e6-52e3-4735-99d5-ee0d86bbc182 with Priority: HIGHEST for action port
[  504.280368] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 413013e6-52e3-4735-99d5-ee0d86bbc182 with Priority: HIGHEST for action port
[  504.281368] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  504.281368] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  504.281597] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  504.281843] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  504.281942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  504.282038] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-413013e6-52e3-4735-99d5-ee0d86bbc182 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  504.282135] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-413013e6-52e3-4735-99d5-ee0d86bbc182" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  504.282200] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-413013e6-52e3-4735-99d5-ee0d86bbc182" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  504.282406] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-413013e6-52e3-4735-99d5-ee0d86bbc182" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  504.282482] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('413013e6-52e3-4735-99d5-ee0d86bbc182',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  504.282817] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg eb390915576a4dd2bdf4f12a031cd2b2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  504.296066] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eb390915576a4dd2bdf4f12a031cd2b2
[  504.436801] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1a6d29cd8f9646f792ca039e2a7501e5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  504.567115] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1a6d29cd8f9646f792ca039e2a7501e5
[  504.625474] env[60890]: INFO networking_nsxv3.common.synchronization [req-4ed6485e-794c-41a6-b10f-312107b0f6bd req-643d6ce4-a921-4a78-9b90-121820eb62a5 tempest-ServerDiagnosticsNegativeTest-1382837043 tempest-ServerDiagnosticsNegativeTest-1382837043-project-member] Enqueued Resource ID: 2559fbfb-9059-4cfb-adf7-22902a0b4012 with Priority: HIGHEST for action security_group_members
[  504.626024] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2559fbfb-9059-4cfb-adf7-22902a0b4012 with Priority: HIGHEST for action security_group_members
[  504.626277] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  504.626397] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  504.626462] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  504.626664] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  504.626744] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  504.626898] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-2559fbfb-9059-4cfb-adf7-22902a0b4012 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  504.626992] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-2559fbfb-9059-4cfb-adf7-22902a0b4012" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  504.627051] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-2559fbfb-9059-4cfb-adf7-22902a0b4012" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  504.627377] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-2559fbfb-9059-4cfb-adf7-22902a0b4012" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  504.627377] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  504.627497] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  504.627533] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  504.627671] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  504.627747] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  504.627832] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('2559fbfb-9059-4cfb-adf7-22902a0b4012',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  504.628031] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d2d920e9d5624626b26366907917e070 in queue reply_156db6c4256a4c58884afce15e6f5340
[  504.669929] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d2d920e9d5624626b26366907917e070
[  504.791418] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-413013e6-52e3-4735-99d5-ee0d86bbc182" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  504.791418] env[60890]: Traceback (most recent call last):
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  504.791418] env[60890]:     timer()
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  504.791418] env[60890]:     cb(*args, **kw)
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  504.791418] env[60890]:     result = function(*args, **kwargs)
[  504.791418] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  504.791418] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  504.791418] env[60890]:     return method(*args, **kwargs)
[  504.791418] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  504.791418] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  504.791418] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  504.791418] env[60890]:     result = self.transport._send(
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  504.791418] env[60890]:     return self._driver.send(target, ctxt, message,
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  504.791418] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  504.791418] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  504.791418] env[60890]:     raise result
[  504.791418] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  504.791418] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  505.137336] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  505.137498] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  505.137498] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  505.137711] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  505.137803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  505.138202] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  505.138202] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  505.138202] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  505.138352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  505.138436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  505.138465] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 2559fbfb-9059-4cfb-adf7-22902a0b4012 already deleted.
[  505.138626] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-2559fbfb-9059-4cfb-adf7-22902a0b4012" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  505.511618] env[60890]: INFO networking_nsxv3.common.synchronization [None req-7b34150c-4a99-48d9-9e45-b356950837d8 None None] Enqueued Resource ID: 01f77a7b-7e1b-4f7a-91ab-85826b9f4a54 with Priority: HIGHEST for action port
[  505.511881] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 01f77a7b-7e1b-4f7a-91ab-85826b9f4a54 with Priority: HIGHEST for action port
[  505.512446] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  505.512566] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  505.512633] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  505.512995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  505.512995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  505.513111] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-01f77a7b-7e1b-4f7a-91ab-85826b9f4a54 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  505.513176] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-01f77a7b-7e1b-4f7a-91ab-85826b9f4a54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  505.513212] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-01f77a7b-7e1b-4f7a-91ab-85826b9f4a54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  505.513521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-01f77a7b-7e1b-4f7a-91ab-85826b9f4a54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  505.513521] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('01f77a7b-7e1b-4f7a-91ab-85826b9f4a54',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  505.513654] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9b291e4fa4da466a9883e0fe947cf037 in queue reply_156db6c4256a4c58884afce15e6f5340
[  505.519011] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9b291e4fa4da466a9883e0fe947cf037
[  506.015647] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-01f77a7b-7e1b-4f7a-91ab-85826b9f4a54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  506.015991] env[60890]: Traceback (most recent call last):
[  506.015991] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  506.015991] env[60890]:     timer()
[  506.015991] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  506.015991] env[60890]:     cb(*args, **kw)
[  506.015991] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  506.015991] env[60890]:     result = function(*args, **kwargs)
[  506.015991] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  506.015991] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  506.016274] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  506.016274] env[60890]:     return method(*args, **kwargs)
[  506.016274] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  506.016274] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  506.016274] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  506.016274] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  506.016274] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  506.016274] env[60890]:     result = self.transport._send(
[  506.016274] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  506.016274] env[60890]:     return self._driver.send(target, ctxt, message,
[  506.016274] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  506.016274] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  506.016274] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  506.016274] env[60890]:     raise result
[  506.016274] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  506.016274] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  506.760558] env[60890]: INFO networking_nsxv3.common.synchronization [None req-72340800-67b7-4b6c-8839-f505b8413f5a None None] Enqueued Resource ID: a071ac03-1ced-421e-a846-4a2d9484e3a1 with Priority: HIGHEST for action port
[  506.760823] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a071ac03-1ced-421e-a846-4a2d9484e3a1 with Priority: HIGHEST for action port
[  506.761379] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  506.761498] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  506.761564] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  506.761770] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  506.761849] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  506.761996] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-a071ac03-1ced-421e-a846-4a2d9484e3a1 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  506.762089] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-a071ac03-1ced-421e-a846-4a2d9484e3a1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  506.762148] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-a071ac03-1ced-421e-a846-4a2d9484e3a1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  506.762336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-a071ac03-1ced-421e-a846-4a2d9484e3a1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  506.762410] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a071ac03-1ced-421e-a846-4a2d9484e3a1',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  506.762644] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 954380b5e1f54241a4a3577a1115cc43 in queue reply_156db6c4256a4c58884afce15e6f5340
[  506.766249] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  506.766511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  506.766674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  506.766937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  506.767112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  506.768069] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  506.768245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  506.768397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  506.768697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  506.772237] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  506.772773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  506.772947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  506.773214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  506.773399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  506.773585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  506.773911] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  506.774147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  506.774286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  506.774705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  506.774903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  506.775186] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  506.775381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  506.775796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  506.776102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  506.776272] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  506.776532] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e3011619b32c4a1c9faec3dea1ca04e8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  506.780602] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 954380b5e1f54241a4a3577a1115cc43
[  506.789146] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e3011619b32c4a1c9faec3dea1ca04e8
[  507.106179] env[60890]: INFO networking_nsxv3.common.synchronization [None req-3c3fafd9-6c6b-4521-9849-3d29171f7d58 None None] Enqueued Resource ID: c039ae3c-42b4-4e66-9c33-1db536db6db5 with Priority: HIGHEST for action port
[  507.106439] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c039ae3c-42b4-4e66-9c33-1db536db6db5 with Priority: HIGHEST for action port
[  507.107048] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  507.107177] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  507.107246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  507.107437] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  507.107516] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  507.107666] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-c039ae3c-42b4-4e66-9c33-1db536db6db5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  507.107752] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-c039ae3c-42b4-4e66-9c33-1db536db6db5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  507.107821] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-c039ae3c-42b4-4e66-9c33-1db536db6db5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  507.107993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-c039ae3c-42b4-4e66-9c33-1db536db6db5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  507.108084] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c039ae3c-42b4-4e66-9c33-1db536db6db5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  507.108261] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 081b2cff82b0468fbd982ead2df85890 in queue reply_156db6c4256a4c58884afce15e6f5340
[  507.139518] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 081b2cff82b0468fbd982ead2df85890
[  507.264919] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-a071ac03-1ced-421e-a846-4a2d9484e3a1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  507.265164] env[60890]: Traceback (most recent call last):
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  507.265235] env[60890]:     timer()
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  507.265235] env[60890]:     cb(*args, **kw)
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  507.265235] env[60890]:     result = function(*args, **kwargs)
[  507.265235] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  507.265235] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  507.265235] env[60890]:     return method(*args, **kwargs)
[  507.265235] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  507.265235] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  507.265235] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  507.265235] env[60890]:     result = self.transport._send(
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  507.265235] env[60890]:     return self._driver.send(target, ctxt, message,
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  507.265235] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  507.265235] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  507.265235] env[60890]:     raise result
[  507.265235] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  507.265235] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  507.279361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  507.279944] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  507.279944] env[60890]: ERROR oslo_messaging.rpc.server 
[  507.611534] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-c039ae3c-42b4-4e66-9c33-1db536db6db5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  507.611534] env[60890]: Traceback (most recent call last):
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  507.611534] env[60890]:     timer()
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  507.611534] env[60890]:     cb(*args, **kw)
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  507.611534] env[60890]:     result = function(*args, **kwargs)
[  507.611534] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  507.611534] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  507.611534] env[60890]:     return method(*args, **kwargs)
[  507.611534] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  507.611534] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  507.611534] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  507.611534] env[60890]:     result = self.transport._send(
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  507.611534] env[60890]:     return self._driver.send(target, ctxt, message,
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  507.611534] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  507.611534] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  507.611534] env[60890]:     raise result
[  507.611534] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  507.611534] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  507.809899] env[60890]: INFO networking_nsxv3.common.synchronization [None req-1a86bacd-4f08-4487-9899-de47d8fa5734 None None] Enqueued Resource ID: 8c396709-6e80-49f2-9ef8-ee00e2c9d549 with Priority: HIGHEST for action port
[  507.810387] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8c396709-6e80-49f2-9ef8-ee00e2c9d549 with Priority: HIGHEST for action port
[  507.810697] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  507.810823] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  507.810888] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  507.811078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  507.811160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  507.811309] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-8c396709-6e80-49f2-9ef8-ee00e2c9d549 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  507.811760] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-8c396709-6e80-49f2-9ef8-ee00e2c9d549" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  507.811760] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-8c396709-6e80-49f2-9ef8-ee00e2c9d549" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  507.811760] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-8c396709-6e80-49f2-9ef8-ee00e2c9d549" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  507.811760] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('8c396709-6e80-49f2-9ef8-ee00e2c9d549',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  507.811926] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg add77f8d3aa445a38807caec932ac943 in queue reply_156db6c4256a4c58884afce15e6f5340
[  507.819955] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg add77f8d3aa445a38807caec932ac943
[  508.141504] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  508.141630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  508.141674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  508.141934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  508.142069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  508.142322] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  508.142471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  508.142585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  508.142823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  508.143138] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  508.143234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  508.143341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  508.143550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  508.143691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  508.143834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  508.144093] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  508.144248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  508.144362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  508.144565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  508.144724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  508.144968] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  508.145117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  508.145233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  508.145434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  508.146995] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  508.147394] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c4e4017a413144a793040491fa8fc4ed in queue reply_156db6c4256a4c58884afce15e6f5340
[  508.161528] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c4e4017a413144a793040491fa8fc4ed
[  508.314052] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-8c396709-6e80-49f2-9ef8-ee00e2c9d549" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  508.314306] env[60890]: Traceback (most recent call last):
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  508.314358] env[60890]:     timer()
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  508.314358] env[60890]:     cb(*args, **kw)
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  508.314358] env[60890]:     result = function(*args, **kwargs)
[  508.314358] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  508.314358] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  508.314358] env[60890]:     return method(*args, **kwargs)
[  508.314358] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  508.314358] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  508.314358] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  508.314358] env[60890]:     result = self.transport._send(
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  508.314358] env[60890]:     return self._driver.send(target, ctxt, message,
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  508.314358] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  508.314358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  508.314358] env[60890]:     raise result
[  508.314955] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  508.314955] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  508.650045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  508.650657] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  508.650657] env[60890]: ERROR oslo_messaging.rpc.server 
[  508.981067] env[60890]: INFO networking_nsxv3.common.synchronization [None req-d5b8f490-c01f-40d2-a9ff-af926c95f576 None None] Enqueued Resource ID: ae16fa22-d765-49aa-98d5-0b3e6420abd0 with Priority: HIGHEST for action port
[  508.981330] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ae16fa22-d765-49aa-98d5-0b3e6420abd0 with Priority: HIGHEST for action port
[  508.981915] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  508.982038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  508.982108] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  508.982309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  508.982394] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  508.982566] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-ae16fa22-d765-49aa-98d5-0b3e6420abd0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  508.982668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-ae16fa22-d765-49aa-98d5-0b3e6420abd0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  508.982731] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-ae16fa22-d765-49aa-98d5-0b3e6420abd0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  508.982930] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-ae16fa22-d765-49aa-98d5-0b3e6420abd0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  508.983006] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae16fa22-d765-49aa-98d5-0b3e6420abd0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  508.983184] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 65a7342cbb634eecb0d1ad306fd09e0a in queue reply_156db6c4256a4c58884afce15e6f5340
[  508.991441] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 65a7342cbb634eecb0d1ad306fd09e0a
[  509.421780] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  509.421979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  509.422107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  509.422450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  509.422514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  509.422798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  509.422964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  509.423070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  509.423282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  509.423521] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  509.423671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  509.423784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  509.423987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  509.424261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  509.424426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  509.424702] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  509.424872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  509.424992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  509.425202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  509.425335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  509.425619] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  509.425831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  509.425831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  509.426036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  509.426172] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  509.426398] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 61db05512ee94a028cbe34640e9c2fd5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  509.434672] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 61db05512ee94a028cbe34640e9c2fd5
[  509.485509] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-ae16fa22-d765-49aa-98d5-0b3e6420abd0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  509.485760] env[60890]: Traceback (most recent call last):
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  509.485829] env[60890]:     timer()
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  509.485829] env[60890]:     cb(*args, **kw)
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  509.485829] env[60890]:     result = function(*args, **kwargs)
[  509.485829] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  509.485829] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  509.485829] env[60890]:     return method(*args, **kwargs)
[  509.485829] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  509.485829] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  509.485829] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  509.485829] env[60890]:     result = self.transport._send(
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  509.485829] env[60890]:     return self._driver.send(target, ctxt, message,
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  509.485829] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  509.485829] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  509.485829] env[60890]:     raise result
[  509.485829] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  509.485829] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  509.934648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  509.935172] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  509.935172] env[60890]: ERROR oslo_messaging.rpc.server 
[  510.410864] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.411170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.411260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.411460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.411608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  510.411869] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.412034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.412151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.412373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.412619] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.412801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.412922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.413130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.413266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  510.413407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  510.413984] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.413984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.413984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.414231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.414285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  510.414491] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.414644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.414845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.414973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.415104] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  510.415339] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fb5b6b0e7a4842f695113eac934d7498 in queue reply_156db6c4256a4c58884afce15e6f5340
[  510.544405] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fb5b6b0e7a4842f695113eac934d7498
[  510.712757] env[60890]: INFO networking_nsxv3.common.synchronization [None req-38825a24-d50e-4557-b459-79cfd9032efd None None] Enqueued Resource ID: 149fc6d9-ff35-48d4-bd40-be827d64da96 with Priority: HIGHEST for action port
[  510.713028] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 149fc6d9-ff35-48d4-bd40-be827d64da96 with Priority: HIGHEST for action port
[  510.713626] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.713756] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.713825] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.714023] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.714114] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  510.714265] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-149fc6d9-ff35-48d4-bd40-be827d64da96 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.714360] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-149fc6d9-ff35-48d4-bd40-be827d64da96" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.714422] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-149fc6d9-ff35-48d4-bd40-be827d64da96" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.714621] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-149fc6d9-ff35-48d4-bd40-be827d64da96" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.714720] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('149fc6d9-ff35-48d4-bd40-be827d64da96',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  510.714904] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c54d22f5dc4e4200bf16c70363824053 in queue reply_156db6c4256a4c58884afce15e6f5340
[  510.721119] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c54d22f5dc4e4200bf16c70363824053
[  510.782588] env[60890]: INFO networking_nsxv3.common.synchronization [None req-747047f9-77ee-4dcb-8907-7e788932ed3a None None] Enqueued Resource ID: ef596e82-d2d7-428b-9638-32410ce7f915 with Priority: HIGHEST for action port
[  510.782882] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ef596e82-d2d7-428b-9638-32410ce7f915 with Priority: HIGHEST for action port
[  510.783428] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.783556] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.783625] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.783822] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.783908] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  510.784093] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-ef596e82-d2d7-428b-9638-32410ce7f915 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  510.784195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-ef596e82-d2d7-428b-9638-32410ce7f915" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  510.784258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-ef596e82-d2d7-428b-9638-32410ce7f915" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  510.784467] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-ef596e82-d2d7-428b-9638-32410ce7f915" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  510.784548] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ef596e82-d2d7-428b-9638-32410ce7f915',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  510.784773] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ed4a7b59067b4c0497d243fd1ccd6000 in queue reply_156db6c4256a4c58884afce15e6f5340
[  510.812401] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ed4a7b59067b4c0497d243fd1ccd6000
[  510.919443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  510.919443] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  510.919443] env[60890]: ERROR oslo_messaging.rpc.server 
[  511.031095] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  511.031290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  511.031414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  511.031678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  511.031821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.032795] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  511.032795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  511.032795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  511.032795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  511.032795] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  511.033072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  511.033072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  511.033255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  511.033393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.033542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.033799] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  511.033953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  511.034066] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  511.034267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  511.034394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.034634] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  511.034811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  511.034926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  511.035132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  511.035265] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  511.035498] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fe239d9363874b9297a7855b37020b6c in queue reply_156db6c4256a4c58884afce15e6f5340
[  511.045694] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fe239d9363874b9297a7855b37020b6c
[  511.216886] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-149fc6d9-ff35-48d4-bd40-be827d64da96" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.217144] env[60890]: Traceback (most recent call last):
[  511.217186] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  511.217186] env[60890]:     timer()
[  511.217186] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  511.217186] env[60890]:     cb(*args, **kw)
[  511.217186] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  511.217186] env[60890]:     result = function(*args, **kwargs)
[  511.217186] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  511.217186] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  511.217186] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  511.217186] env[60890]:     return method(*args, **kwargs)
[  511.217186] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  511.217186] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  511.217186] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  511.217186] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  511.217186] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  511.217186] env[60890]:     result = self.transport._send(
[  511.217186] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  511.217186] env[60890]:     return self._driver.send(target, ctxt, message,
[  511.217663] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  511.217663] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  511.217663] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  511.217663] env[60890]:     raise result
[  511.217663] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  511.217663] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  511.274293] env[60890]: INFO networking_nsxv3.common.synchronization [None req-0ff5c2bb-fedf-47b1-9795-a42a6dae4149 None None] Enqueued Resource ID: 5657c01d-704c-4f05-96ad-3667f9fc91fd with Priority: HIGHEST for action port
[  511.274562] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5657c01d-704c-4f05-96ad-3667f9fc91fd with Priority: HIGHEST for action port
[  511.275165] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  511.275290] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  511.275380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  511.275589] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  511.275684] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.275848] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-5657c01d-704c-4f05-96ad-3667f9fc91fd taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  511.275951] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-5657c01d-704c-4f05-96ad-3667f9fc91fd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  511.276033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-5657c01d-704c-4f05-96ad-3667f9fc91fd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  511.276239] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-5657c01d-704c-4f05-96ad-3667f9fc91fd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  511.276322] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5657c01d-704c-4f05-96ad-3667f9fc91fd',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  511.276502] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b9fdcfb95fd84e898f3d4679e53e9ad6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  511.282050] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b9fdcfb95fd84e898f3d4679e53e9ad6
[  511.287384] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-ef596e82-d2d7-428b-9638-32410ce7f915" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.287618] env[60890]: Traceback (most recent call last):
[  511.287666] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  511.287666] env[60890]:     timer()
[  511.287666] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  511.287666] env[60890]:     cb(*args, **kw)
[  511.287785] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  511.287785] env[60890]:     result = function(*args, **kwargs)
[  511.287785] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  511.287785] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  511.287785] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  511.287785] env[60890]:     return method(*args, **kwargs)
[  511.287785] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  511.287785] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  511.287785] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  511.287785] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  511.287785] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  511.287785] env[60890]:     result = self.transport._send(
[  511.287785] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  511.287785] env[60890]:     return self._driver.send(target, ctxt, message,
[  511.287785] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  511.287785] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  511.287785] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  511.287785] env[60890]:     raise result
[  511.287785] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  511.287785] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  511.537797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  511.538309] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  511.538309] env[60890]: ERROR oslo_messaging.rpc.server 
[  511.779390] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-5657c01d-704c-4f05-96ad-3667f9fc91fd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  511.779638] env[60890]: Traceback (most recent call last):
[  511.779689] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  511.779689] env[60890]:     timer()
[  511.779689] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  511.779689] env[60890]:     cb(*args, **kw)
[  511.779689] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  511.779689] env[60890]:     result = function(*args, **kwargs)
[  511.779689] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  511.779689] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  511.779689] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  511.779689] env[60890]:     return method(*args, **kwargs)
[  511.779689] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  511.779689] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  511.779689] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  511.779689] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  511.779689] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  511.779689] env[60890]:     result = self.transport._send(
[  511.779689] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  511.779689] env[60890]:     return self._driver.send(target, ctxt, message,
[  511.780199] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  511.780199] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  511.780199] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  511.780199] env[60890]:     raise result
[  511.780199] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  511.780199] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  512.239939] env[60890]: INFO networking_nsxv3.common.synchronization [None req-60d130e6-4f0c-4880-8fc6-42638812d96b None None] Enqueued Resource ID: 34de03c3-b5f4-41b3-a21a-ed74fa902d02 with Priority: HIGHEST for action port
[  512.240042] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 34de03c3-b5f4-41b3-a21a-ed74fa902d02 with Priority: HIGHEST for action port
[  512.240576] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.240699] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.240766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.240956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.241039] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.241186] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-34de03c3-b5f4-41b3-a21a-ed74fa902d02 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.241280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-34de03c3-b5f4-41b3-a21a-ed74fa902d02" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.241342] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-34de03c3-b5f4-41b3-a21a-ed74fa902d02" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.241542] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-34de03c3-b5f4-41b3-a21a-ed74fa902d02" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.241625] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('34de03c3-b5f4-41b3-a21a-ed74fa902d02',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  512.241805] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg bb64015a01ea4d309e40c3f8b1d79cf0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  512.248611] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bb64015a01ea4d309e40c3f8b1d79cf0
[  512.340832] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.340920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.340966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.341224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.341363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.341617] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.341771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.341888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.342155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.342338] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.342487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.342636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.342852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.342990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.343890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.343890] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.343890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.343890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.343890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.344151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.345041] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.345041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.345041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.345041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.345041] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  512.345206] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d43425b8e3c14e4783e0aadd4602ad16 in queue reply_156db6c4256a4c58884afce15e6f5340
[  512.442687] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d43425b8e3c14e4783e0aadd4602ad16
[  512.659876] env[60890]: INFO networking_nsxv3.common.synchronization [req-50675964-e257-4b9a-8b5e-255db72b5f0e req-bff4ecd0-953b-4d30-8f80-c5f782b5b189 tempest-DeleteServersAdminTestJSON-570518421 tempest-DeleteServersAdminTestJSON-570518421-project-member] Enqueued Resource ID: 86656952-d254-40e4-a34d-b4f64574ede0 with Priority: HIGHEST for action security_group_members
[  512.660423] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 86656952-d254-40e4-a34d-b4f64574ede0 with Priority: HIGHEST for action security_group_members
[  512.660738] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.660867] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.660937] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.661145] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.661279] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.661386] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-86656952-d254-40e4-a34d-b4f64574ede0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.661485] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.661548] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.661753] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.661895] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  512.662053] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  512.662053] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  512.662183] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  512.662261] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.662348] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('86656952-d254-40e4-a34d-b4f64574ede0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  512.662534] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg dc71b864147a451f8067e388c2a9a22c in queue reply_156db6c4256a4c58884afce15e6f5340
[  512.743449] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-34de03c3-b5f4-41b3-a21a-ed74fa902d02" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.743688] env[60890]: Traceback (most recent call last):
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  512.743741] env[60890]:     timer()
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  512.743741] env[60890]:     cb(*args, **kw)
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  512.743741] env[60890]:     result = function(*args, **kwargs)
[  512.743741] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  512.743741] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  512.743741] env[60890]:     return method(*args, **kwargs)
[  512.743741] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  512.743741] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  512.743741] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  512.743741] env[60890]:     result = self.transport._send(
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  512.743741] env[60890]:     return self._driver.send(target, ctxt, message,
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  512.743741] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  512.743741] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  512.743741] env[60890]:     raise result
[  512.744357] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  512.744357] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  512.770298] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dc71b864147a451f8067e388c2a9a22c
[  512.846797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  512.847275] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  512.847275] env[60890]: ERROR oslo_messaging.rpc.server 
[  513.164996] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.165179] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.165258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.165468] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.165567] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.165816] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.165949] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.166023] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.166226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.166311] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.166399] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 86656952-d254-40e4-a34d-b4f64574ede0 already deleted.
[  513.166480] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.362728] env[60890]: INFO networking_nsxv3.common.synchronization [None req-74324780-6bc1-45ab-9f3e-222cdb904812 None None] Enqueued Resource ID: b31df545-3d8f-45f3-93f2-f3b6cc32a74b with Priority: HIGHEST for action port
[  513.363057] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b31df545-3d8f-45f3-93f2-f3b6cc32a74b with Priority: HIGHEST for action port
[  513.363681] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.363823] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.363904] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.364148] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.364215] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.364369] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-b31df545-3d8f-45f3-93f2-f3b6cc32a74b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.364461] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-b31df545-3d8f-45f3-93f2-f3b6cc32a74b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.364520] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-b31df545-3d8f-45f3-93f2-f3b6cc32a74b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.364804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-b31df545-3d8f-45f3-93f2-f3b6cc32a74b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.364896] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b31df545-3d8f-45f3-93f2-f3b6cc32a74b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  513.365108] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ae159129e4c3442385ff3d8bffd304ae in queue reply_156db6c4256a4c58884afce15e6f5340
[  513.552193] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ae159129e4c3442385ff3d8bffd304ae
[  513.868134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-b31df545-3d8f-45f3-93f2-f3b6cc32a74b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.868376] env[60890]: Traceback (most recent call last):
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  513.868376] env[60890]:     timer()
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  513.868376] env[60890]:     cb(*args, **kw)
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  513.868376] env[60890]:     result = function(*args, **kwargs)
[  513.868376] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  513.868376] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  513.868376] env[60890]:     return method(*args, **kwargs)
[  513.868376] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  513.868376] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  513.868376] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  513.868376] env[60890]:     result = self.transport._send(
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  513.868376] env[60890]:     return self._driver.send(target, ctxt, message,
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  513.868376] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  513.868376] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  513.868376] env[60890]:     raise result
[  513.868376] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  513.868376] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  513.968278] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.968483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.968607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.969014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.969014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.969289] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.969427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.969540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.969747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.969986] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.970168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.970245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.970442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.970579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.970720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.970968] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.971118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.971875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.971875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.971875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  513.971875] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  513.972065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  513.972065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  513.972260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  513.972391] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  513.972624] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 44731d10a99648bd9b2a8ffa585101bc in queue reply_156db6c4256a4c58884afce15e6f5340
[  514.003272] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 44731d10a99648bd9b2a8ffa585101bc
[  514.360160] env[60890]: INFO networking_nsxv3.common.synchronization [None req-1e87d1d2-696f-4b3f-9959-c40af49ce315 None None] Enqueued Resource ID: eac136c7-dcb1-4abc-a669-557b8b7c57b3 with Priority: HIGHEST for action port
[  514.360160] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: eac136c7-dcb1-4abc-a669-557b8b7c57b3 with Priority: HIGHEST for action port
[  514.360160] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.360160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.360160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.360472] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.360472] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.360532] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-eac136c7-dcb1-4abc-a669-557b8b7c57b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.360894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-eac136c7-dcb1-4abc-a669-557b8b7c57b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.360894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-eac136c7-dcb1-4abc-a669-557b8b7c57b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.360894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-eac136c7-dcb1-4abc-a669-557b8b7c57b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.361027] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eac136c7-dcb1-4abc-a669-557b8b7c57b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  514.361135] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 82c155cfdc864805acc46bcd02e96472 in queue reply_156db6c4256a4c58884afce15e6f5340
[  514.399851] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 82c155cfdc864805acc46bcd02e96472
[  514.475480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  514.476064] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  514.476064] env[60890]: ERROR oslo_messaging.rpc.server 
[  514.656248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.656248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.656248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.656248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.656248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.656248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.656248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.656248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.656684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.656684] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.656763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.656862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.657100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.657266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.657394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.657645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.657791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.657908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.658111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.658257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.658851] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.659071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.659221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.659561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.659706] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  514.659972] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3da256c617c948d2bf31e34e20b0b3ce in queue reply_156db6c4256a4c58884afce15e6f5340
[  514.674598] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3da256c617c948d2bf31e34e20b0b3ce
[  514.741013] env[60890]: INFO networking_nsxv3.common.synchronization [None req-e52529e1-2dbd-4c7a-b1fe-afa75075b684 tempest-VolumesAdminNegativeTest-2086354477 tempest-VolumesAdminNegativeTest-2086354477-project-member] Enqueued Resource ID: b371c264-efdf-44ab-9a75-9ff23ee9391e with Priority: HIGHEST for action security_group_rules
[  514.741497] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b371c264-efdf-44ab-9a75-9ff23ee9391e with Priority: HIGHEST for action security_group_rules
[  514.741808] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.741928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.741990] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.742186] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.742267] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.742413] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-b371c264-efdf-44ab-9a75-9ff23ee9391e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.742520] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.742615] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.742748] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.742824] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('b371c264-efdf-44ab-9a75-9ff23ee9391e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  514.743001] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9bae2df856b349789614fa40bb207236 in queue reply_156db6c4256a4c58884afce15e6f5340
[  514.757176] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9bae2df856b349789614fa40bb207236
[  514.822099] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.822302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.822427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.822977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.823143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.823429] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.823587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.823701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.823921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.824174] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.824321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.824433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.824648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.824804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.824953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.825251] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.825367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.825459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.825661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.825864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.826046] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.826194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.826642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.826874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.827035] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  514.827280] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9a6e40b1e10c4dacaa80cf36891d19d3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  514.832703] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9a6e40b1e10c4dacaa80cf36891d19d3
[  514.863560] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-eac136c7-dcb1-4abc-a669-557b8b7c57b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.863814] env[60890]: Traceback (most recent call last):
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  514.863875] env[60890]:     timer()
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  514.863875] env[60890]:     cb(*args, **kw)
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  514.863875] env[60890]:     result = function(*args, **kwargs)
[  514.863875] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  514.863875] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  514.863875] env[60890]:     return method(*args, **kwargs)
[  514.863875] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  514.863875] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  514.863875] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  514.863875] env[60890]:     result = self.transport._send(
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  514.863875] env[60890]:     return self._driver.send(target, ctxt, message,
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  514.863875] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  514.863875] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  514.863875] env[60890]:     raise result
[  514.863875] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  514.863875] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  514.913479] env[60890]: INFO networking_nsxv3.common.synchronization [None req-a0027281-7616-4a33-bf0a-669837723a03 tempest-VolumesAdminNegativeTest-2086354477 tempest-VolumesAdminNegativeTest-2086354477-project-member] Enqueued Resource ID: b371c264-efdf-44ab-9a75-9ff23ee9391e with Priority: HIGHEST for action security_group_rules
[  514.913991] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b371c264-efdf-44ab-9a75-9ff23ee9391e with Priority: HIGHEST for action security_group_rules
[  514.914301] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.914426] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  514.914492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  514.916278] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  514.916382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  514.917055] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-b371c264-efdf-44ab-9a75-9ff23ee9391e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  514.917161] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.164319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  515.164319] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  515.164319] env[60890]: ERROR oslo_messaging.rpc.server 
[  515.252175] env[60890]: INFO networking_nsxv3.common.synchronization [req-1bdaeff2-4879-4c08-bae9-b1e0444b2a04 req-7ca24a87-3185-45af-8a04-68a017e5938b tempest-FloatingIPsAssociationTestJSON-1051505831 tempest-FloatingIPsAssociationTestJSON-1051505831-project-member] Enqueued Resource ID: 014641c6-6987-45ee-8ee5-ed1bf025b019 with Priority: HIGHEST for action security_group_members
[  515.252175] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 014641c6-6987-45ee-8ee5-ed1bf025b019 with Priority: HIGHEST for action security_group_members
[  515.252175] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.252175] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-014641c6-6987-45ee-8ee5-ed1bf025b019 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-014641c6-6987-45ee-8ee5-ed1bf025b019" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-014641c6-6987-45ee-8ee5-ed1bf025b019" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-014641c6-6987-45ee-8ee5-ed1bf025b019" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.252175] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.252175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.252175] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('014641c6-6987-45ee-8ee5-ed1bf025b019',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  515.252175] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 99f544c7012e4a768b75d20e8de3b4ea in queue reply_156db6c4256a4c58884afce15e6f5340
[  515.259012] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 99f544c7012e4a768b75d20e8de3b4ea
[  515.263611] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.263926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.264105] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.264383] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.264564] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.264844] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: b371c264-efdf-44ab-9a75-9ff23ee9391e already deleted.
[  515.265032] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.265729] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.266051] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.266286] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('b371c264-efdf-44ab-9a75-9ff23ee9391e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  515.266643] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ab5c96cc83d5473e996ed21e8382c06e in queue reply_156db6c4256a4c58884afce15e6f5340
[  515.279871] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ab5c96cc83d5473e996ed21e8382c06e
[  515.284134] env[60890]: INFO networking_nsxv3.common.synchronization [None req-29b6ddda-ada8-4e06-84be-620199bb544f None None] Enqueued Resource ID: faa09542-de0d-4bf7-85f5-3429c9865f77 with Priority: HIGHEST for action port
[  515.284134] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: faa09542-de0d-4bf7-85f5-3429c9865f77 with Priority: HIGHEST for action port
[  515.284134] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.284134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.284134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.284134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.284134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.284134] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-faa09542-de0d-4bf7-85f5-3429c9865f77 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.284134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-faa09542-de0d-4bf7-85f5-3429c9865f77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.286104] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-faa09542-de0d-4bf7-85f5-3429c9865f77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.286104] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-faa09542-de0d-4bf7-85f5-3429c9865f77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.286104] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('faa09542-de0d-4bf7-85f5-3429c9865f77',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  515.286104] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a144d357368847f2aed5dcedf5af25ee in queue reply_156db6c4256a4c58884afce15e6f5340
[  515.290901] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a144d357368847f2aed5dcedf5af25ee
[  515.329950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  515.330456] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  515.330456] env[60890]: ERROR oslo_messaging.rpc.server 
[  515.645999] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.646194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.646317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.646574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.646719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.646999] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.647168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.647288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.647497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.647738] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.647892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.648020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.648222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.648357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.648498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.648747] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.648884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.649035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.649196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.649327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.649560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.649709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.649821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.650025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.650154] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  515.650382] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 23b70ac4267e49fdae5d7c009d9f5533 in queue reply_156db6c4256a4c58884afce15e6f5340
[  515.656305] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 23b70ac4267e49fdae5d7c009d9f5533
[  515.756590] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.756590] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.756590] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 014641c6-6987-45ee-8ee5-ed1bf025b019 already deleted.
[  515.756590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-014641c6-6987-45ee-8ee5-ed1bf025b019" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.788367] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  515.788698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  515.788698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  515.788845] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  515.790010] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.790010] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: b371c264-efdf-44ab-9a75-9ff23ee9391e already deleted.
[  515.790010] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.790010] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-faa09542-de0d-4bf7-85f5-3429c9865f77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  515.790010] env[60890]: Traceback (most recent call last):
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  515.790010] env[60890]:     timer()
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  515.790010] env[60890]:     cb(*args, **kw)
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  515.790010] env[60890]:     result = function(*args, **kwargs)
[  515.790010] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  515.790010] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  515.790010] env[60890]:     return method(*args, **kwargs)
[  515.790010] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  515.790010] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  515.790010] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  515.790010] env[60890]:     result = self.transport._send(
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  515.790010] env[60890]:     return self._driver.send(target, ctxt, message,
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  515.790010] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  515.790010] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  515.790010] env[60890]:     raise result
[  515.790010] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  515.790010] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  516.153040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  516.153595] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  516.153595] env[60890]: ERROR oslo_messaging.rpc.server 
[  516.322344] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.322542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.322679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.323379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.323538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.323804] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.324137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.324137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.324337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.324583] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.325236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.325236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.325236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.325236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.325441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.325562] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.325754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.325859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.326073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.326262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.326427] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-62cc40e9-d174-4184-9b4c-5adba6d515a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.326661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.326737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.326883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.327007] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('62cc40e9-d174-4184-9b4c-5adba6d515a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  516.327278] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 550ab37247de4d49905c885a9036891a in queue reply_156db6c4256a4c58884afce15e6f5340
[  516.334096] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 550ab37247de4d49905c885a9036891a
[  516.363428] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.363631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.363756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.364042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.364191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.364452] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.364843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.364843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.365030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.365351] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.365532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.365659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.365861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.366001] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.366147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.366396] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.366548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.366662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.366868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.367020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.367310] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.367510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.367653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.367866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.367999] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  516.368263] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 260071a05ccc406c81821772a84926b8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  516.378923] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 260071a05ccc406c81821772a84926b8
[  516.496053] env[60890]: INFO networking_nsxv3.common.synchronization [req-02e78f33-4e46-4ed0-9c24-586f4b9d17ff req-5d926caf-6f26-4298-a654-13182fcde8a3 tempest-SecurityGroupRulesNegativeTestJSON-2016580145 tempest-SecurityGroupRulesNegativeTestJSON-2016580145-project-member] Enqueued Resource ID: 8e0b1a3e-42b4-44d1-bceb-9ac95add7d07 with Priority: HIGHEST for action security_group_rules
[  516.496661] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8e0b1a3e-42b4-44d1-bceb-9ac95add7d07 with Priority: HIGHEST for action security_group_rules
[  516.496920] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.497456] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.497456] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.497456] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.497456] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.497620] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  516.497694] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  516.497694] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  516.497882] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  516.498832] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('8e0b1a3e-42b4-44d1-bceb-9ac95add7d07',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  516.498832] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3cdf6117e6ca47679bb398147154393f in queue reply_156db6c4256a4c58884afce15e6f5340
[  516.520522] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3cdf6117e6ca47679bb398147154393f
[  516.830328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-62cc40e9-d174-4184-9b4c-5adba6d515a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  516.830844] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  516.830844] env[60890]: ERROR oslo_messaging.rpc.server 
[  516.870968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  516.871465] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  516.871465] env[60890]: ERROR oslo_messaging.rpc.server 
[  517.019335] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.019522] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.019567] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.019790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.019922] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.020051] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 8e0b1a3e-42b4-44d1-bceb-9ac95add7d07 already deleted.
[  517.020160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.076181] env[60890]: INFO networking_nsxv3.common.synchronization [None req-5e991f2d-6b17-4243-a37a-a43c5a91d80d None None] Enqueued Resource ID: 83e7039e-61de-4daa-989a-8efaac3d143b with Priority: HIGHEST for action port
[  517.076857] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 83e7039e-61de-4daa-989a-8efaac3d143b with Priority: HIGHEST for action port
[  517.077149] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.077288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.077348] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.077558] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.077680] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.077858] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-83e7039e-61de-4daa-989a-8efaac3d143b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.077991] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-83e7039e-61de-4daa-989a-8efaac3d143b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.078053] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-83e7039e-61de-4daa-989a-8efaac3d143b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.078287] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-83e7039e-61de-4daa-989a-8efaac3d143b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.078366] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('83e7039e-61de-4daa-989a-8efaac3d143b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  517.078547] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b5a79055c5694a03bf864ae988322693 in queue reply_156db6c4256a4c58884afce15e6f5340
[  517.085154] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b5a79055c5694a03bf864ae988322693
[  517.210126] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.210453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.210575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.210842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.210986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.211472] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.211646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.211766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.212060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.212311] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.212500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.212629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.212907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.213000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.213149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.213403] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.213553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.213675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.213881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.214038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.214318] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.214474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.214587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.214864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.214999] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  517.215232] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg dd761ef202ea48158ef6c4ef9849d62a in queue reply_156db6c4256a4c58884afce15e6f5340
[  517.226580] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dd761ef202ea48158ef6c4ef9849d62a
[  517.324650] env[60890]: INFO networking_nsxv3.common.synchronization [req-d8acbc98-722d-4cca-bf4e-148ee0510705 req-32d97b9f-7c99-4476-a912-769078b32c80 tempest-FloatingIPsAssociationNegativeTestJSON-680885566 tempest-FloatingIPsAssociationNegativeTestJSON-680885566-project-member] Enqueued Resource ID: d701bc90-6679-4fa4-b21b-e638b2b64596 with Priority: HIGHEST for action security_group_members
[  517.324650] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d701bc90-6679-4fa4-b21b-e638b2b64596 with Priority: HIGHEST for action security_group_members
[  517.324650] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.324650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.324650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.324650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.324650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.324650] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-d701bc90-6679-4fa4-b21b-e638b2b64596 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.324650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-d701bc90-6679-4fa4-b21b-e638b2b64596" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.324650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-d701bc90-6679-4fa4-b21b-e638b2b64596" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.325114] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-d701bc90-6679-4fa4-b21b-e638b2b64596" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.325114] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.325114] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.325114] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.325258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.325258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.325317] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('d701bc90-6679-4fa4-b21b-e638b2b64596',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  517.325544] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a7f0df660cb0457383b42b3cb314e14b in queue reply_156db6c4256a4c58884afce15e6f5340
[  517.333797] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a7f0df660cb0457383b42b3cb314e14b
[  517.580974] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-83e7039e-61de-4daa-989a-8efaac3d143b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.580974] env[60890]: Traceback (most recent call last):
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  517.580974] env[60890]:     timer()
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  517.580974] env[60890]:     cb(*args, **kw)
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  517.580974] env[60890]:     result = function(*args, **kwargs)
[  517.580974] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  517.580974] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  517.580974] env[60890]:     return method(*args, **kwargs)
[  517.580974] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  517.580974] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  517.580974] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  517.580974] env[60890]:     result = self.transport._send(
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  517.580974] env[60890]:     return self._driver.send(target, ctxt, message,
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  517.580974] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  517.580974] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  517.580974] env[60890]:     raise result
[  517.580974] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  517.580974] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  517.591677] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.591882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.592028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.592821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.592821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.592821] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.592821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.593055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.594368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.594708] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.594889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.595008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.595325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.595473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.595631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.595907] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.596054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.596173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.596387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.596523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.596760] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.596945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.597035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.597245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.597382] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  517.598356] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4169ff722a4f49a397412a5a1cd1c740 in queue reply_156db6c4256a4c58884afce15e6f5340
[  517.717224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  517.717722] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  517.717722] env[60890]: ERROR oslo_messaging.rpc.server 
[  517.723330] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4169ff722a4f49a397412a5a1cd1c740
[  517.827208] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.827377] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.827447] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.827651] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.827746] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.827937] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  517.828066] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  517.828126] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  517.828278] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  517.828363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  517.828438] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: d701bc90-6679-4fa4-b21b-e638b2b64596 already deleted.
[  517.828512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-d701bc90-6679-4fa4-b21b-e638b2b64596" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  518.107282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  518.107282] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  518.107282] env[60890]: ERROR oslo_messaging.rpc.server 
[  518.745631] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  518.745907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  518.745965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  518.746217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  518.746373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  518.746636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  518.746803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  518.746904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  518.747121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  518.747405] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  518.747521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  518.747640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  518.747852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  518.747995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  518.749205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  518.749526] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  518.749727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  518.749884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  518.750141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  518.750313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  518.750816] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  518.751013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  518.751165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  518.752105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  518.752105] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  518.752105] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1a4707c12bda46f4b227a527492ebfc3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  518.759445] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1a4707c12bda46f4b227a527492ebfc3
[  519.254229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  519.254814] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  519.254814] env[60890]: ERROR oslo_messaging.rpc.server 
[  519.443181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.443181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.443181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.443181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.443181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.443181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.443181] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  519.443181] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 499af2870604420ca68afb5bb6f6b289 in queue reply_156db6c4256a4c58884afce15e6f5340
[  519.449949] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 499af2870604420ca68afb5bb6f6b289
[  519.565577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.565577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.565577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.565577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.565577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.566319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.566724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.568158] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.568158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.568158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.568158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.568158] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  519.568158] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8deab6cf633642859a70ffc54f4a76f0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  519.570055] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.570266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.570440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.570772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.570914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.571314] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.571537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.572563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.572563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.573027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.573027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.573195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.573461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.573661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.573867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.574201] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.574495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.574703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.575008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.575201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.575494] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.575715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.575948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.576281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.578341] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  519.578638] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 225dc4be4dc240fcb72ce4a193accf23 in queue reply_156db6c4256a4c58884afce15e6f5340
[  519.579695] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8deab6cf633642859a70ffc54f4a76f0
[  519.584508] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 225dc4be4dc240fcb72ce4a193accf23
[  519.696782] env[60890]: INFO networking_nsxv3.common.synchronization [None req-334a8fb8-90be-400f-abea-c84a31a5ce23 None None] Enqueued Resource ID: fe423ca6-02dc-45bb-94f9-d9dc9436ebb2 with Priority: HIGHEST for action port
[  519.697042] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: fe423ca6-02dc-45bb-94f9-d9dc9436ebb2 with Priority: HIGHEST for action port
[  519.697611] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.697741] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.697811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.698005] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.698091] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.698241] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-fe423ca6-02dc-45bb-94f9-d9dc9436ebb2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.698335] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-fe423ca6-02dc-45bb-94f9-d9dc9436ebb2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.698399] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-fe423ca6-02dc-45bb-94f9-d9dc9436ebb2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.698592] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-fe423ca6-02dc-45bb-94f9-d9dc9436ebb2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.698672] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fe423ca6-02dc-45bb-94f9-d9dc9436ebb2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  519.698851] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9e4edd71fda14e6c9391f46a554918de in queue reply_156db6c4256a4c58884afce15e6f5340
[  519.704116] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9e4edd71fda14e6c9391f46a554918de
[  519.939164] env[60890]: INFO networking_nsxv3.common.synchronization [req-4ed6485e-794c-41a6-b10f-312107b0f6bd req-08052595-04ae-4e1a-8982-f191790fbf6d tempest-ServerDiagnosticsNegativeTest-1382837043 tempest-ServerDiagnosticsNegativeTest-1382837043-project-member] Enqueued Resource ID: 2559fbfb-9059-4cfb-adf7-22902a0b4012 with Priority: HIGHEST for action security_group_members
[  519.939705] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2559fbfb-9059-4cfb-adf7-22902a0b4012 with Priority: HIGHEST for action security_group_members
[  519.940085] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.940246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.940330] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.940566] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.940665] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.940853] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-2559fbfb-9059-4cfb-adf7-22902a0b4012 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.940961] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-2559fbfb-9059-4cfb-adf7-22902a0b4012" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.941260] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-2559fbfb-9059-4cfb-adf7-22902a0b4012" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.941260] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-2559fbfb-9059-4cfb-adf7-22902a0b4012" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.941400] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  519.941400] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  519.941476] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  519.941594] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  519.941668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.941763] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('2559fbfb-9059-4cfb-adf7-22902a0b4012',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  519.941950] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8e63ef00f29340b7ae4de8149f276861 in queue reply_156db6c4256a4c58884afce15e6f5340
[  519.948872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  519.949389] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  519.949389] env[60890]: ERROR oslo_messaging.rpc.server 
[  520.053904] env[60890]: INFO networking_nsxv3.common.synchronization [req-fa05baee-4514-4bb3-ab39-ebb0b2fdc414 req-fc8a93e4-7db2-40f0-bcab-56db4cc8f2ef tempest-MigrationsAdminTest-320299187 tempest-MigrationsAdminTest-320299187-project-member] Enqueued Resource ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b with Priority: HIGHEST for action security_group_members
[  520.053904] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b with Priority: HIGHEST for action security_group_members
[  520.053904] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.054118] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.054118] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.054239] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.054301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.054442] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-18ba6c5b-4074-4f30-b957-61d36273ed7b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.054550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.054593] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.054927] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.055001] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.055051] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.055146] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.055578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.055578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.055578] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('18ba6c5b-4074-4f30-b957-61d36273ed7b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  520.055578] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 26b59e6d15aa4e2b9f65bccf003a9fad in queue reply_156db6c4256a4c58884afce15e6f5340
[  520.069757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  520.070266] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  520.070266] env[60890]: ERROR oslo_messaging.rpc.server 
[  520.073427] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 26b59e6d15aa4e2b9f65bccf003a9fad
[  520.082463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  520.084175] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  520.084175] env[60890]: ERROR oslo_messaging.rpc.server 
[  520.118329] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8e63ef00f29340b7ae4de8149f276861
[  520.204038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-fe423ca6-02dc-45bb-94f9-d9dc9436ebb2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.204038] env[60890]: Traceback (most recent call last):
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  520.204038] env[60890]:     timer()
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  520.204038] env[60890]:     cb(*args, **kw)
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  520.204038] env[60890]:     result = function(*args, **kwargs)
[  520.204038] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  520.204038] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  520.204038] env[60890]:     return method(*args, **kwargs)
[  520.204038] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  520.204038] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  520.204038] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  520.204038] env[60890]:     result = self.transport._send(
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  520.204038] env[60890]:     return self._driver.send(target, ctxt, message,
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  520.204038] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  520.204038] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  520.204038] env[60890]:     raise result
[  520.204038] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  520.204038] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  520.301239] env[60890]: INFO networking_nsxv3.common.synchronization [None req-39f186f2-70db-4551-8c82-73126c80711b None None] Enqueued Resource ID: 6952091f-2f88-4411-92e9-af5b006b35d3 with Priority: HIGHEST for action port
[  520.301476] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6952091f-2f88-4411-92e9-af5b006b35d3 with Priority: HIGHEST for action port
[  520.302028] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.302157] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.302223] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.302428] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.302509] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.302681] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-6952091f-2f88-4411-92e9-af5b006b35d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.302775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-6952091f-2f88-4411-92e9-af5b006b35d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.302835] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-6952091f-2f88-4411-92e9-af5b006b35d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.303036] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-6952091f-2f88-4411-92e9-af5b006b35d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.303116] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6952091f-2f88-4411-92e9-af5b006b35d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  520.303302] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 62c51cbf19a64384a5485b1528549047 in queue reply_156db6c4256a4c58884afce15e6f5340
[  520.312971] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 62c51cbf19a64384a5485b1528549047
[  520.343008] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 97 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.343008] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.343008] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.343141] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.343445] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.343445] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock all taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 100 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.343532] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.343562] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.343683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.343901] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_ports_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  520.344109] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 728cc4ff33ba478e86f88a88216371b7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  520.347437] env[60890]: INFO networking_nsxv3.common.synchronization [None req-ed0648dd-75e6-42f4-9b89-7b7dbe3afb9a None None] Enqueued Resource ID: 65fa517f-d4b2-4d3b-82c9-b92ea7aa94ed with Priority: HIGHEST for action port
[  520.347763] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 65fa517f-d4b2-4d3b-82c9-b92ea7aa94ed with Priority: HIGHEST for action port
[  520.348337] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.348483] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.348571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.348925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.348925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.349098] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-65fa517f-d4b2-4d3b-82c9-b92ea7aa94ed taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.349217] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-65fa517f-d4b2-4d3b-82c9-b92ea7aa94ed" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.349297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-65fa517f-d4b2-4d3b-82c9-b92ea7aa94ed" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.349576] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-65fa517f-d4b2-4d3b-82c9-b92ea7aa94ed" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.349688] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65fa517f-d4b2-4d3b-82c9-b92ea7aa94ed',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  520.349907] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7458a106828e4f48a00971c6a6fccf93 in queue reply_156db6c4256a4c58884afce15e6f5340
[  520.381237] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7458a106828e4f48a00971c6a6fccf93
[  520.381735] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 728cc4ff33ba478e86f88a88216371b7
[  520.444065] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.444231] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.444300] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.444514] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.444619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.445022] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.445022] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.445022] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.445195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.445226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.446357] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 2559fbfb-9059-4cfb-adf7-22902a0b4012 already deleted.
[  520.446357] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-2559fbfb-9059-4cfb-adf7-22902a0b4012" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.501356] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.501569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.501708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.501943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.502092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.502322] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.502459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.502570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.502768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.503008] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.503159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.503303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.503459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.503588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.503725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.504108] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.504195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.504241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.504430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.504553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.504826] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.504972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.505299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.505432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.505557] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  520.505846] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0d9af6dd70194a13963f049cc9a377f3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  520.518741] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0d9af6dd70194a13963f049cc9a377f3
[  520.536862] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.537085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.537213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.537476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.537613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.538650] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.538650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.538650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.538650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.538650] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.538914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.538914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.539018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.539154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.539297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.539545] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.539699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.539814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.540099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.540196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.541191] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.541191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.541191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.541191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.541191] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  520.541466] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fd8bf167bfc14f9eabe21fcebcfa9947 in queue reply_156db6c4256a4c58884afce15e6f5340
[  520.552879] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fd8bf167bfc14f9eabe21fcebcfa9947
[  520.559482] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.559674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.559674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.562408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.562408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.562408] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.562408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.562408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.562408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.562408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.562408] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b already deleted.
[  520.562408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.642083] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.642271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.642390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.642662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.642802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.643059] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.643206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.643317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.643526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.643762] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.643905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.644025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.644229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.644362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.644499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.644764] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.644914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.645024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.645225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.645348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.645579] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  520.645750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  520.645864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  520.646065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  520.646190] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  520.646646] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a89ce97f5d9645f199d88c22f6dec634 in queue reply_156db6c4256a4c58884afce15e6f5340
[  520.652930] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a89ce97f5d9645f199d88c22f6dec634
[  520.806315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-6952091f-2f88-4411-92e9-af5b006b35d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.806315] env[60890]: Traceback (most recent call last):
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  520.806315] env[60890]:     timer()
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  520.806315] env[60890]:     cb(*args, **kw)
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  520.806315] env[60890]:     result = function(*args, **kwargs)
[  520.806315] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  520.806315] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  520.806315] env[60890]:     return method(*args, **kwargs)
[  520.806315] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  520.806315] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  520.806315] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  520.806315] env[60890]:     result = self.transport._send(
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  520.806315] env[60890]:     return self._driver.send(target, ctxt, message,
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  520.806315] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  520.806315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  520.806315] env[60890]:     raise result
[  520.806315] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  520.806315] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  520.848890] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_groups_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  520.849111] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7903991cf2ff44a5acd7a7e3697be72d in queue reply_156db6c4256a4c58884afce15e6f5340
[  520.851668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-65fa517f-d4b2-4d3b-82c9-b92ea7aa94ed" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  520.851891] env[60890]: Traceback (most recent call last):
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  520.851941] env[60890]:     timer()
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  520.851941] env[60890]:     cb(*args, **kw)
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  520.851941] env[60890]:     result = function(*args, **kwargs)
[  520.851941] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  520.851941] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  520.851941] env[60890]:     return method(*args, **kwargs)
[  520.851941] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  520.851941] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  520.851941] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  520.851941] env[60890]:     result = self.transport._send(
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  520.851941] env[60890]:     return self._driver.send(target, ctxt, message,
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  520.851941] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  520.851941] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  520.851941] env[60890]:     raise result
[  520.852708] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  520.852708] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  520.860340] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7903991cf2ff44a5acd7a7e3697be72d
[  521.008855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  521.008855] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  521.008855] env[60890]: ERROR oslo_messaging.rpc.server 
[  521.046026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  521.046026] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  521.046026] env[60890]: ERROR oslo_messaging.rpc.server 
[  521.151438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  521.152082] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  521.152082] env[60890]: ERROR oslo_messaging.rpc.server 
[  521.252138] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.252347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.252472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.252783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.252930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.253183] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.253329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.253462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.253646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.253880] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.254077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.254216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.254421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.254557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.254720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.254971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.255117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.255228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.255429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.255650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.255787] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.255937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.256835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.257558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.257958] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  521.257958] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3df12ab312cd4d5b9e68f68af657c99b in queue reply_156db6c4256a4c58884afce15e6f5340
[  521.267665] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3df12ab312cd4d5b9e68f68af657c99b
[  521.351213] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_qos_policies_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  521.351441] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 122e2c6390154336b0742f88a4c3598e in queue reply_156db6c4256a4c58884afce15e6f5340
[  521.363749] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 122e2c6390154336b0742f88a4c3598e
[  521.707829] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.708180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.708180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.708431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.708576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.708831] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.708987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.709102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.709310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.709546] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.709696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.709811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.710056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.710149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.710289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.710533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.710683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.710800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.711383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.711383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.711383] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.711563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.711602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.711786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.711914] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  521.712157] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ec03299900d246f992dc35d98a0401cd in queue reply_156db6c4256a4c58884afce15e6f5340
[  521.718187] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ec03299900d246f992dc35d98a0401cd
[  521.759732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  521.760672] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  521.760672] env[60890]: ERROR oslo_messaging.rpc.server 
[  521.853826] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Inventory metadata is going to be refreshed.
[  521.853931] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Segment.
[  521.994095] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.994095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.994233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.994425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.994565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.995041] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.995041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.995194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.995731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.995731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.995731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.995911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.996063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.996579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.996579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.996861] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.996861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.997003] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.997340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.997340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  521.997527] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  521.997673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  521.997848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  521.998469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  521.998469] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  521.998669] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0f12af8ab5514bfc9838e963f3b1f728 in queue reply_156db6c4256a4c58884afce15e6f5340
[  522.003921] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.003921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.003921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.003921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.003921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.003921] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.005665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.005797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.006073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.006325] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.006482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.006599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.006817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.007051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.007225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.007478] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.007630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.007743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.007949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.008092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.008326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.008478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.008591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.008796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.008928] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  522.009157] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ef50ac28348e49518336e200f1695e38 in queue reply_156db6c4256a4c58884afce15e6f5340
[  522.026807] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0f12af8ab5514bfc9838e963f3b1f728
[  522.037666] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ef50ac28348e49518336e200f1695e38
[  522.214378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  522.214972] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  522.214972] env[60890]: ERROR oslo_messaging.rpc.server 
[  522.512975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  522.512975] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  522.512975] env[60890]: ERROR oslo_messaging.rpc.server 
[  522.520768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  522.520768] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  522.520768] env[60890]: ERROR oslo_messaging.rpc.server 
[  522.587637] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 118 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.588033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.588206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.588519] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.605385] env[60890]: WARNING networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider [-] Duplicate resource with OS_ID: 992 ID: cl2-zone-992
[  522.606080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.607440] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:SegmentPort.
[  522.639608] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock SegmentPort taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 120 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.639990] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.640222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.640524] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.658902] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.659858] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:SegmentPort Is:96.
[  522.660114] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:SegmentPort Are:{'519bac90-d40c-494c-8eb9-6c53e239f10a', '32332759-ca87-4ef6-955a-325ad700200d', '03fdd0ff-044d-4309-905b-d51a3bab1b81', '55f117f0-ba76-491f-b5c5-7441d3ff7787', '9435dc4a-ead0-4afd-a099-668739ae29f4', '66d98031-6842-491c-9961-c1b4d0fd5d53', '0f875a82-d52b-4617-86ef-09c6a8aeccbb', '2126120d-06ba-46a5-a74c-77f3b18345ab', '17b157f7-bf15-4a5c-a271-9af72764322d', '474946e7-99a3-4290-959e-f2dc1645b999', '4fb6c40c-bf99-4b55-9f6b-ca451ded5287', '7e152039-3f45-45d1-b2a1-c7acb717f68b', '1ca72486-d800-43c5-907b-725bb958b3a3', '2a1bc88a-7641-4017-874c-f15ee8ec3621', 'd357f4a7-7fe1-4677-a32d-b2a6c5780592', '99c860bb-14da-4e95-b94e-d77d15224b7a', 'fb2c7956-257f-4d1d-8dbb-83b5350798c3', 'c4585efa-40da-4b2f-81a8-523efec14826', '7e3eabc0-1383-412e-a875-c4e73daf6808', 'f599de71-cb96-446c-b853-bfcdc671260e', '31125a4c-2800-482d-8420-53a798a29c02', '0cb4488c-6859-4b6c-b322-dc290dbf0f8f', '3e63d897-dbf9-4401-a6d7-1650e7c0ecdf', '62510d77-c23b-4044-a0f1-0f38a92caf2c', '11ab945e-c9e0-4540-8ddf-a57fc4983384', '0ecbd240-2457-42ca-b168-e67fa16f56b3', '156f2622-5bca-4370-8d7a-06130b978cf7', '6f69240a-5122-4e7e-87af-f288a47b5ea9', 'e758e5fd-8995-41a7-9fe7-7ce7894fb7aa', '1e891f4b-00a4-481c-a00c-0320f9e75e62', 'd43c8245-514d-4c94-80b7-c12dfe9a6434', '088ac320-7bf8-4ebf-9318-5c12a3de2e62', '78c543de-6fc1-464e-b0a5-b0db23f19be3', 'ea8f9597-af66-4849-94da-7357f4969707', '889eac8e-299a-4c2b-a8e4-819f8f54a261', '08b26599-acd3-4729-9ae4-7cef241ebca7', '046ad10e-2286-420f-b6f3-0d598b3fcc03', 'bc6fc679-542b-4d2b-838e-70db009ad31a', '64487042-dc1f-4bbb-8b1c-0ff06c270b8d', '45968fc0-6e74-43f7-862b-4043c3249370', '39825d23-3b58-4cdb-8fa7-5a9ff493c789', '2eea5bcc-a0a3-4963-a879-5bf0c1c38f60', '2763deba-ab9e-48b4-b5a8-f7082a993850', 'e9ad7926-1d42-4ff9-aab3-d520de5ae40e', '33359e95-dbc3-4199-9367-e6bd0129468e', '32f3a7a3-308f-4fc7-8898-ed5ad6161587', '88452c84-34f9-49bc-aa27-a6d503e47fa4', 'e1979b45-d8d8-4076-a153-fa64ba78b7c0', 'ace6f305-6029-421e-89d3-bcca9fd2264a', '06f70962-82ab-4995-b7ca-a341783a2c6b', 'ac9ba9d7-c9f6-482f-9cbe-4ac9659c3194', '2da5eafe-fcd1-44dd-b2aa-c1bc16ff3d23', '34dd8f0f-dd08-44fa-9865-37de198ce337', '02696c3f-3dd0-499c-9baa-8a8cf21253eb', 'a385dc0d-17c5-49e9-9685-8285fc885a67', '1b4cdb06-1445-4694-a8a7-25090c7c8982', '74ea2df8-d554-4541-bfb6-8a8fad4756ea', '28285b72-9ad5-4703-950f-d20766e1592d', 'c4c889cc-a303-49e8-b5da-c71c873abb02', '885d71ab-3cf8-44a1-98fe-fedf69f9f021', 'e798c106-4155-4e94-90dc-5591d8705c9e', 'f5808565-bb46-4620-9c27-c24fe7a3d1b6', 'a161c3dd-1d1b-49c2-816f-07c0cc3fe929', '6865e1e1-9000-4b73-b41e-4a4753755582', '271514e3-602e-4224-bfb9-99943b868420', '9674a2da-6ae2-430a-80a2-67bab8177b7e', '8e1ea150-2eec-4dac-80e6-7d1daeaf351d', 'dcde1b76-b76a-4b6b-bbc8-099834069781', '925b62f2-60ee-49b1-974b-66ad3d724498', 'e9d5bf46-3b2c-4088-a6bf-07b65f970ae4', 'f467c1f7-4700-4387-bb2a-249f697bbb6c', '7fcb3b5c-ae38-468f-aecc-8e3293f03e66', 'c4d845cb-64ec-4e2f-87a3-29dcdf4f4b2b', 'c8e81773-32ca-437e-9971-50cef2a264ea', 'd73acfa0-7f7a-4f3e-98ae-e9e2b940b462', 'a367d71c-4063-4a70-9c6c-4f60da50879e', '3903c4b9-6919-4b40-9189-942790370190', 'f1f40c6a-f2c1-4a74-94e2-cdf2e32e24cd', '6040c7d6-38fb-48bc-9161-2b810f0ffb1a', 'fc8312bb-e44e-4d35-b825-f0e74adf2260', 'fa51a02a-4b3e-479b-b703-759a3babfeb1', 'f1446fa4-56d0-44ca-8222-7ed499af9ee1', 'c9f3dd72-2950-4d13-9d1f-d1d7538d8150', 'e3e3d0bf-2787-48ab-9720-f7cb2a5d4ce2', 'f00d4feb-c74c-4e36-ab50-7989164b3eba', 'c5200c6e-cf74-4b1f-8377-44fbfd9e43ac', '48e0c1c5-8a6b-4724-a313-2b27b5d0073d', '194b348c-e895-4f8d-ab94-c33cb6abd8b7', 'c6537245-d078-41cd-b50c-1df8377c8fd6', '0cec1ac8-30f8-403a-a9b6-97dd88f439b7', 'ca735a43-ae10-4d73-851b-23c32b6e90a2', '0e5e87d8-236e-443c-b231-531694a623f3', '10d09f8d-b8e6-456d-85e8-1ecfa1dd206f', 'ed65677d-a1b5-4cb2-a8c2-fd130551ab54', '85bd5f38-c811-4e68-8e48-2bed178d2a9a', 'e937c7a1-6075-47df-be05-bce50cae3909'} {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  522.660426] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Rules).
[  522.706439] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.706439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.706439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.706702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.706876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.707137] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.707288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.707698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.707698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.707844] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.707990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.709410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.709939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.709939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.710290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.712069] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.712069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.712069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.712069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.712069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.712069] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.712069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.712069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.712069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.712069] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  522.713001] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a0581ea911184679b9e40c7a49a9218a in queue reply_156db6c4256a4c58884afce15e6f5340
[  522.719506] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a0581ea911184679b9e40c7a49a9218a
[  522.827729] env[60890]: INFO networking_nsxv3.common.synchronization [None req-b02e9a77-fc0a-4c64-8d1b-a13e8ea6442f None None] Enqueued Resource ID: 13666244-658d-4252-bc12-dd0766588714 with Priority: HIGHEST for action port
[  522.827991] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 13666244-658d-4252-bc12-dd0766588714 with Priority: HIGHEST for action port
[  522.828620] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.828753] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.828819] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.829121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.829121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.829242] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-13666244-658d-4252-bc12-dd0766588714 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.829362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-13666244-658d-4252-bc12-dd0766588714" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.829398] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-13666244-658d-4252-bc12-dd0766588714" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.829572] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-13666244-658d-4252-bc12-dd0766588714" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.829649] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('13666244-658d-4252-bc12-dd0766588714',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  522.829833] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg dd4b6e1c99da407fb381f7f2d2955d4c in queue reply_156db6c4256a4c58884afce15e6f5340
[  522.835733] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dd4b6e1c99da407fb381f7f2d2955d4c
[  522.913065] env[60890]: INFO networking_nsxv3.common.synchronization [req-1bfcdc5b-4f92-428e-bca2-d342e4dce45f req-33372605-563f-4289-a7ff-057a3aece746 tempest-SecurityGroupRulesNegativeTestJSON-2016580145 tempest-SecurityGroupRulesNegativeTestJSON-2016580145-project-member] Enqueued Resource ID: 8e0b1a3e-42b4-44d1-bceb-9ac95add7d07 with Priority: HIGHEST for action security_group_rules
[  522.914553] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8e0b1a3e-42b4-44d1-bceb-9ac95add7d07 with Priority: HIGHEST for action security_group_rules
[  522.914909] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.915114] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.915185] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.915437] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.916209] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  522.916444] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  522.916597] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  522.916679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  522.916872] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  522.917706] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('8e0b1a3e-42b4-44d1-bceb-9ac95add7d07',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  522.917938] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 106c4975ddc243e7b05f9e54cf31fe7b in queue reply_156db6c4256a4c58884afce15e6f5340
[  522.968569] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 106c4975ddc243e7b05f9e54cf31fe7b
[  523.215787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.216146] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.216146] env[60890]: ERROR oslo_messaging.rpc.server 
[  523.280690] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.280889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.281592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.281592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.281592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.281736] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.281813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.281916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.282125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.282357] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.282588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.282784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.283105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.283328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.283585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.284453] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.284635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.284774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.284979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.285113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.285349] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.285493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.285609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.285892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.286037] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  523.286272] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3f0ebef34d2d4b0d881dec59c41d0735 in queue reply_156db6c4256a4c58884afce15e6f5340
[  523.314408] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3f0ebef34d2d4b0d881dec59c41d0735
[  523.332289] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-13666244-658d-4252-bc12-dd0766588714" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.332458] env[60890]: Traceback (most recent call last):
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  523.332506] env[60890]:     timer()
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  523.332506] env[60890]:     cb(*args, **kw)
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  523.332506] env[60890]:     result = function(*args, **kwargs)
[  523.332506] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  523.332506] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  523.332506] env[60890]:     return method(*args, **kwargs)
[  523.332506] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  523.332506] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  523.332506] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  523.332506] env[60890]:     result = self.transport._send(
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  523.332506] env[60890]:     return self._driver.send(target, ctxt, message,
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  523.332506] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  523.332506] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  523.332506] env[60890]:     raise result
[  523.333182] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.333182] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.373845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.373845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.373845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.374062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.374198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.375160] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.375160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.375160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.375160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.375160] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.375401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.375435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.375653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.375733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.375869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.376234] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.376379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.376493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.376689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.376810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.377037] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.377226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.377283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.377476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.377600] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  523.377859] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1d055c4613bc4495809f96e543945bcd in queue reply_156db6c4256a4c58884afce15e6f5340
[  523.381770] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.381770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.381770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.381770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.381770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.381770] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.381770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.381770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.381770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.382134] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.382302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.382431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.382664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.382807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.382959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.383199] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.383351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.383527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.383739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.384568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.384568] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.384568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.384568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.384568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.384811] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  523.384926] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a9271fb0f21549a78ff982b2f31b974f in queue reply_156db6c4256a4c58884afce15e6f5340
[  523.397622] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1d055c4613bc4495809f96e543945bcd
[  523.397622] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a9271fb0f21549a78ff982b2f31b974f
[  523.404115] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.404115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.404115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.404115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.426025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.427452] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Rules Remote IP Prefix).
[  523.453902] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.454123] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.454229] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.454556] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.454932] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.455250] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 8e0b1a3e-42b4-44d1-bceb-9ac95add7d07 already deleted.
[  523.455392] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-8e0b1a3e-42b4-44d1-bceb-9ac95add7d07" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.789455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.789986] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.789986] env[60890]: ERROR oslo_messaging.rpc.server 
[  523.888040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.888040] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.888040] env[60890]: ERROR oslo_messaging.rpc.server 
[  523.888906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.889146] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  523.889146] env[60890]: ERROR oslo_messaging.rpc.server 
[  523.962328] env[60890]: INFO networking_nsxv3.common.synchronization [None req-9f7871ca-8e94-410b-ba94-778a015599de None None] Enqueued Resource ID: 694ef6f3-8311-4d30-b380-565724c209ce with Priority: HIGHEST for action port
[  523.962506] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 694ef6f3-8311-4d30-b380-565724c209ce with Priority: HIGHEST for action port
[  523.962777] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.962905] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.962971] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.963174] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.963255] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  523.963408] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-694ef6f3-8311-4d30-b380-565724c209ce taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  523.963505] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-694ef6f3-8311-4d30-b380-565724c209ce" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  523.963564] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-694ef6f3-8311-4d30-b380-565724c209ce" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  523.963758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-694ef6f3-8311-4d30-b380-565724c209ce" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  523.963826] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('694ef6f3-8311-4d30-b380-565724c209ce',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  523.964038] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f43e90eb7b0b4ffdb330053e6f2c3f52 in queue reply_156db6c4256a4c58884afce15e6f5340
[  523.987503] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f43e90eb7b0b4ffdb330053e6f2c3f52
[  524.089535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.089734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.089881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.090146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.090259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.090630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.090719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.090783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.091001] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.091327] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.091453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.091555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.091734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.092220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.092220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.092315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.092460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.092501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.092698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.092827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.093057] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.093205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.093313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.093554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.093632] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  524.093854] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b42f9cd7bc144764afee62bb4436ae90 in queue reply_156db6c4256a4c58884afce15e6f5340
[  524.107775] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b42f9cd7bc144764afee62bb4436ae90
[  524.107775] env[60890]: INFO networking_nsxv3.common.synchronization [None req-829a9ca1-e785-464c-a98c-b532d8c41f75 None None] Enqueued Resource ID: 19fd0d27-80ec-4cb1-8f11-6e7d6fe23024 with Priority: HIGHEST for action port
[  524.107775] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 19fd0d27-80ec-4cb1-8f11-6e7d6fe23024 with Priority: HIGHEST for action port
[  524.107775] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.107775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.107775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.107775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.107775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.107775] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-19fd0d27-80ec-4cb1-8f11-6e7d6fe23024 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.107775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-19fd0d27-80ec-4cb1-8f11-6e7d6fe23024" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.107775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-19fd0d27-80ec-4cb1-8f11-6e7d6fe23024" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.107775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-19fd0d27-80ec-4cb1-8f11-6e7d6fe23024" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.107775] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('19fd0d27-80ec-4cb1-8f11-6e7d6fe23024',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  524.107775] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 92d2069e5f4644668f8f525d568fadfc in queue reply_156db6c4256a4c58884afce15e6f5340
[  524.125364] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 92d2069e5f4644668f8f525d568fadfc
[  524.240119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.240119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.240119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.240119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.240119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.240119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.240119] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  524.240119] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5e846f564af04973baac320a2524a7c0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  524.343313] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.344025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.344025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.344025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.344175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.344372] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.344530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.344685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.344847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.345085] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.346092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.346092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.346092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.346092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.346092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.346311] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.346408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.346689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.346797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.346932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.347170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.347320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.347432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.347635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.347767] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  524.348036] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 25aee0f5236e4d91a7ed50148bc97699 in queue reply_156db6c4256a4c58884afce15e6f5340
[  524.349151] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5e846f564af04973baac320a2524a7c0
[  524.352086] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 25aee0f5236e4d91a7ed50148bc97699
[  524.465945] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-694ef6f3-8311-4d30-b380-565724c209ce" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.466195] env[60890]: Traceback (most recent call last):
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  524.466261] env[60890]:     timer()
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  524.466261] env[60890]:     cb(*args, **kw)
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  524.466261] env[60890]:     result = function(*args, **kwargs)
[  524.466261] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  524.466261] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  524.466261] env[60890]:     return method(*args, **kwargs)
[  524.466261] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  524.466261] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  524.466261] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  524.466261] env[60890]:     result = self.transport._send(
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  524.466261] env[60890]:     return self._driver.send(target, ctxt, message,
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  524.466261] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  524.466261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  524.466261] env[60890]:     raise result
[  524.466261] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  524.466261] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  524.495586] env[60890]: INFO networking_nsxv3.common.synchronization [None req-e2fbd97e-b3a9-4de4-82c1-bc40a64b5e6a tempest-ServerActionsTestOtherB-1179403528 tempest-ServerActionsTestOtherB-1179403528-project-member] Enqueued Resource ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 with Priority: HIGHEST for action security_group_rules
[  524.495586] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 with Priority: HIGHEST for action security_group_rules
[  524.495998] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.496352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.496352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.496820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.496820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.496932] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-0cd25894-1483-4857-b8b7-2f23e1ed9869 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.497048] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.497143] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.497378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.497498] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('0cd25894-1483-4857-b8b7-2f23e1ed9869',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  524.497745] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3377e89176aa4740a264edf59cd23506 in queue reply_156db6c4256a4c58884afce15e6f5340
[  524.506160] env[60890]: INFO networking_nsxv3.common.synchronization [req-fd28f446-8abc-486d-92de-f9c6c3c33d37 req-927e5b0f-41c7-43ea-bbcb-2d64dabad2dd tempest-ServersAdminNegativeTestJSON-1199624387 tempest-ServersAdminNegativeTestJSON-1199624387-project-member] Enqueued Resource ID: 807320f7-85fa-40dd-aa93-4f085c5d7eb7 with Priority: HIGHEST for action security_group_members
[  524.509766] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 807320f7-85fa-40dd-aa93-4f085c5d7eb7 with Priority: HIGHEST for action security_group_members
[  524.509766] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.509766] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-807320f7-85fa-40dd-aa93-4f085c5d7eb7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-807320f7-85fa-40dd-aa93-4f085c5d7eb7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-807320f7-85fa-40dd-aa93-4f085c5d7eb7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-807320f7-85fa-40dd-aa93-4f085c5d7eb7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.509766] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.509766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.509766] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('807320f7-85fa-40dd-aa93-4f085c5d7eb7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  524.509766] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c5e1390a68b04df39f57a4d16998237f in queue reply_156db6c4256a4c58884afce15e6f5340
[  524.527534] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3377e89176aa4740a264edf59cd23506
[  524.536249] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c5e1390a68b04df39f57a4d16998237f
[  524.597121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  524.597682] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  524.597682] env[60890]: ERROR oslo_messaging.rpc.server 
[  524.610095] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-19fd0d27-80ec-4cb1-8f11-6e7d6fe23024" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.610304] env[60890]: Traceback (most recent call last):
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  524.610349] env[60890]:     timer()
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  524.610349] env[60890]:     cb(*args, **kw)
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  524.610349] env[60890]:     result = function(*args, **kwargs)
[  524.610349] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  524.610349] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  524.610349] env[60890]:     return method(*args, **kwargs)
[  524.610349] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  524.610349] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  524.610349] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  524.610349] env[60890]:     result = self.transport._send(
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  524.610349] env[60890]:     return self._driver.send(target, ctxt, message,
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  524.610349] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  524.610349] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  524.610349] env[60890]:     raise result
[  524.610950] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  524.610950] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  524.624181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.624375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.624493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.624798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.624943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.625199] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.625348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.625458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.625674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.625908] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.626052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.626162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.626360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.626493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.626677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.626867] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.627039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.627158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.627357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.627483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.627723] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.627867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.627976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.628264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.628404] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  524.628625] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3089cafc9e344d3ea933ce62b6d6f8e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  524.634250] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3089cafc9e344d3ea933ce62b6d6f8e0
[  524.730321] env[60890]: INFO networking_nsxv3.common.synchronization [None req-948fc02a-f71a-448a-a74a-f590a80eff59 None None] Enqueued Resource ID: b6f215f9-1aac-4633-9f48-c6afc3ee9fb8 with Priority: HIGHEST for action port
[  524.730585] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b6f215f9-1aac-4633-9f48-c6afc3ee9fb8 with Priority: HIGHEST for action port
[  524.731399] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.731521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.731586] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.731793] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.731875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.732054] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-b6f215f9-1aac-4633-9f48-c6afc3ee9fb8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  524.732151] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-b6f215f9-1aac-4633-9f48-c6afc3ee9fb8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  524.732211] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-b6f215f9-1aac-4633-9f48-c6afc3ee9fb8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  524.732411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-b6f215f9-1aac-4633-9f48-c6afc3ee9fb8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  524.732487] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b6f215f9-1aac-4633-9f48-c6afc3ee9fb8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  524.732667] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 14fd6cf81dfd4eb68cf627d7eebcec54 in queue reply_156db6c4256a4c58884afce15e6f5340
[  524.743056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  524.743810] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  524.743810] env[60890]: ERROR oslo_messaging.rpc.server 
[  524.752396] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 14fd6cf81dfd4eb68cf627d7eebcec54
[  524.851298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  524.851819] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  524.851819] env[60890]: ERROR oslo_messaging.rpc.server 
[  525.019294] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.019294] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.019294] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 807320f7-85fa-40dd-aa93-4f085c5d7eb7 already deleted.
[  525.019294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-807320f7-85fa-40dd-aa93-4f085c5d7eb7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.020035] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.020035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.020035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.020035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.020035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.020233] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 already deleted.
[  525.020233] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.131332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  525.131845] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  525.131845] env[60890]: ERROR oslo_messaging.rpc.server 
[  525.181778] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.181971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.182091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.182355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.182490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.182753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.182900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.183010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.183241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.183485] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.183634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.183744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.183947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.184145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.184287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.184527] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.184679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.184796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.184995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.185119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.185343] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.185484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.185591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.185829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.185992] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  525.186178] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a2acd48c463a4472a67feaced26988d9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  525.193921] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2acd48c463a4472a67feaced26988d9
[  525.234386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-b6f215f9-1aac-4633-9f48-c6afc3ee9fb8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.234631] env[60890]: Traceback (most recent call last):
[  525.234681] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  525.234681] env[60890]:     timer()
[  525.234681] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  525.234681] env[60890]:     cb(*args, **kw)
[  525.234681] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  525.234681] env[60890]:     result = function(*args, **kwargs)
[  525.234681] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  525.234681] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  525.234681] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  525.234681] env[60890]:     return method(*args, **kwargs)
[  525.234681] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  525.234681] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  525.234681] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  525.234681] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  525.235046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  525.235046] env[60890]:     result = self.transport._send(
[  525.235046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  525.235046] env[60890]:     return self._driver.send(target, ctxt, message,
[  525.235046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  525.235046] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  525.235046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  525.235046] env[60890]:     raise result
[  525.235046] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  525.235046] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  525.467152] env[60890]: INFO networking_nsxv3.common.synchronization [None req-184de774-52b0-49a7-b3d9-36551c901702 tempest-ServerActionsTestOtherB-1179403528 tempest-ServerActionsTestOtherB-1179403528-project-member] Enqueued Resource ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 with Priority: HIGHEST for action security_group_rules
[  525.467530] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 with Priority: HIGHEST for action security_group_rules
[  525.467824] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.467947] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.468036] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.468250] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.468333] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.468500] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-0cd25894-1483-4857-b8b7-2f23e1ed9869 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.468585] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.468650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.468803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.469177] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('0cd25894-1483-4857-b8b7-2f23e1ed9869',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  525.469177] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ff086ca954bc4adaaf12ebbb60a34faf in queue reply_156db6c4256a4c58884afce15e6f5340
[  525.483509] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ff086ca954bc4adaaf12ebbb60a34faf
[  525.525189] env[60890]: INFO networking_nsxv3.common.synchronization [req-7aa552ef-de94-4607-83a3-a08f5006a11d req-9fb02794-66c3-4c69-b687-7bc708b0417d tempest-ServerDiagnosticsTest-269439887 tempest-ServerDiagnosticsTest-269439887-project-member] Enqueued Resource ID: 1bfc53ee-d4bc-45f5-9de4-febc384e1ca2 with Priority: HIGHEST for action security_group_members
[  525.526313] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 1bfc53ee-d4bc-45f5-9de4-febc384e1ca2 with Priority: HIGHEST for action security_group_members
[  525.527095] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.527265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.527382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.527648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.527793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.528102] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.528260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.528431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.528589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.528827] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.528976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.529087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.529319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.529411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.530240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.530240] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.530240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.530240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.530240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.530395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.530554] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.530679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.530780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.530986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.531132] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  525.531334] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 36a5e1fcb23248d38038c7bae860f215 in queue reply_156db6c4256a4c58884afce15e6f5340
[  525.532268] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.532353] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.532461] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.532639] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.532639] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.532845] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.532845] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.532927] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.533090] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.533228] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.533323] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.533383] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.533518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.533701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.533701] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('1bfc53ee-d4bc-45f5-9de4-febc384e1ca2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  525.533829] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d7074002d09c4d5faa399b942d227712 in queue reply_156db6c4256a4c58884afce15e6f5340
[  525.539987] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 36a5e1fcb23248d38038c7bae860f215
[  525.545043] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d7074002d09c4d5faa399b942d227712
[  525.606066] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.606265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.606371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.606651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.606773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.607021] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.607168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.607277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.607477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.607714] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.607877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.607960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.608179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.608362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.608441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.608804] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.608942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.608942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.609133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.609279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.609634] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.609634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.609764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.609925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.610080] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  525.610264] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg aadd41c18485408c84766d3b739a7546 in queue reply_156db6c4256a4c58884afce15e6f5340
[  525.624215] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg aadd41c18485408c84766d3b739a7546
[  525.689347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  525.689872] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  525.689872] env[60890]: ERROR oslo_messaging.rpc.server 
[  525.732555] env[60890]: INFO networking_nsxv3.common.synchronization [None req-6bf7c8e0-e7cf-4b28-8966-0865d3045a40 None None] Enqueued Resource ID: b1a0a533-757e-4cc1-a7b5-5d3504751c42 with Priority: HIGHEST for action port
[  525.732840] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b1a0a533-757e-4cc1-a7b5-5d3504751c42 with Priority: HIGHEST for action port
[  525.733425] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.733942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.734032] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.734221] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.734306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.734467] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-b1a0a533-757e-4cc1-a7b5-5d3504751c42 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.734560] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-b1a0a533-757e-4cc1-a7b5-5d3504751c42" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.734617] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-b1a0a533-757e-4cc1-a7b5-5d3504751c42" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.734870] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-b1a0a533-757e-4cc1-a7b5-5d3504751c42" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.734962] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b1a0a533-757e-4cc1-a7b5-5d3504751c42',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  525.735138] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d04ade74e8e7453fbc3dd96092c9c09d in queue reply_156db6c4256a4c58884afce15e6f5340
[  525.743973] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d04ade74e8e7453fbc3dd96092c9c09d
[  525.990225] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  525.990386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  525.990453] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  525.990678] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  525.990777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  525.990874] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 already deleted.
[  525.990959] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.036290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.036795] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.036795] env[60890]: ERROR oslo_messaging.rpc.server 
[  526.037912] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.038050] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.038115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.038316] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.038404] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.038631] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.038792] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.038876] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.039035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.039250] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.039250] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 1bfc53ee-d4bc-45f5-9de4-febc384e1ca2 already deleted.
[  526.039336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.119670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.120177] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.120177] env[60890]: ERROR oslo_messaging.rpc.server 
[  526.198954] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.199254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.199254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.199490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.199640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.199900] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.200070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.200189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.200410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.200659] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.200811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.200925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.201200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.201358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.201504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.201750] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.201902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.202016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.202223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.202355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.202761] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.202941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.203096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.203274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.203539] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  526.203641] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 23f71c140aa94744be7332df4ffa4e48 in queue reply_156db6c4256a4c58884afce15e6f5340
[  526.218365] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 23f71c140aa94744be7332df4ffa4e48
[  526.238080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-b1a0a533-757e-4cc1-a7b5-5d3504751c42" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.238278] env[60890]: Traceback (most recent call last):
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  526.238358] env[60890]:     timer()
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  526.238358] env[60890]:     cb(*args, **kw)
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  526.238358] env[60890]:     result = function(*args, **kwargs)
[  526.238358] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  526.238358] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  526.238358] env[60890]:     return method(*args, **kwargs)
[  526.238358] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  526.238358] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  526.238358] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  526.238358] env[60890]:     result = self.transport._send(
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  526.238358] env[60890]:     return self._driver.send(target, ctxt, message,
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  526.238358] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  526.238358] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  526.238358] env[60890]:     raise result
[  526.238358] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.238358] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.245628] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.245845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.245977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.247047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.247204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.247470] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.247626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.247743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.248254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.248511] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.248666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.248783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.248988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.249127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.249271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.249508] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.250919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.251069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.253379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.253379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.253379] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.253379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.255442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.260122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.260122] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  526.260122] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ba1e90eb60e84785b7b932426b064bc0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  526.265356] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.265356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.265356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.265356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.265844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.265921] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.266085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.266197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.266453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.267038] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.268475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.268985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.269400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.269694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.270048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.270395] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.270716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.270979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.271465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.271778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.272155] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.272387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.272568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.272922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.273268] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  526.274346] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fa620dc968294886bab4d2fdafcd6ae4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  526.275144] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ba1e90eb60e84785b7b932426b064bc0
[  526.279257] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fa620dc968294886bab4d2fdafcd6ae4
[  526.294369] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules Remote IP Prefix) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 970 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.294713] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.294942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.295305] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.308883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.317562] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Address Group.
[  526.321922] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.322453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.322453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.322669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.322830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.323133] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.323321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.323461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.323713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.323991] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.324224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.324388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.324665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.324935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.325004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.325265] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.325653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.325653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.325852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.326020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.326275] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.326455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.326588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.327485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.327485] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  526.327485] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cac2ccf6d4dd4391a58b138361225ce4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  526.341670] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cac2ccf6d4dd4391a58b138361225ce4
[  526.408075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.408667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.408667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.408994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.409178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.409480] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.409680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.409837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.410115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.411119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.411385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.411556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.411801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.412045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.412260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.412658] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.412888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.413044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.413289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.413521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.413803] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.414015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.414186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.414430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.414665] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  526.414898] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bc0d1b0c20244e7388e63010f98cfb83 in queue reply_156db6c4256a4c58884afce15e6f5340
[  526.421289] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bc0d1b0c20244e7388e63010f98cfb83
[  526.535993] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.536300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.536340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.536575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.536726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.536973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.537123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.537233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.537437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.537684] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.537817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.537926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.538123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.538258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.538397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.538641] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.538790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.538900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.539095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.539219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.539448] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  526.539591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  526.539700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  526.539956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  526.540126] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  526.540362] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e0de814ee8e54ca18837c3500b195eb1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  526.584730] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e0de814ee8e54ca18837c3500b195eb1
[  526.706169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.706854] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.706854] env[60890]: ERROR oslo_messaging.rpc.server 
[  526.764104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.764602] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.764602] env[60890]: ERROR oslo_messaging.rpc.server 
[  526.776982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.777482] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.777482] env[60890]: ERROR oslo_messaging.rpc.server 
[  526.836140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.836140] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.836140] env[60890]: ERROR oslo_messaging.rpc.server 
[  526.917286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.917696] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  526.917696] env[60890]: ERROR oslo_messaging.rpc.server 
[  527.033424] env[60890]: INFO networking_nsxv3.common.synchronization [None req-8a62b378-c600-4774-a77f-7f9e27221f0a None None] Enqueued Resource ID: 8b90bc12-de92-48aa-ab9c-83d0e44199ad with Priority: HIGHEST for action port
[  527.033674] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8b90bc12-de92-48aa-ab9c-83d0e44199ad with Priority: HIGHEST for action port
[  527.034244] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.034375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.034441] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.034641] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.034739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.034895] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-8b90bc12-de92-48aa-ab9c-83d0e44199ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.034990] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-8b90bc12-de92-48aa-ab9c-83d0e44199ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.035048] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-8b90bc12-de92-48aa-ab9c-83d0e44199ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.035714] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-8b90bc12-de92-48aa-ab9c-83d0e44199ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.035714] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('8b90bc12-de92-48aa-ab9c-83d0e44199ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  527.035714] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 69a329e62c56464fac338eba1fbde0b5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  527.047373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.047813] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.047813] env[60890]: ERROR oslo_messaging.rpc.server 
[  527.051733] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 69a329e62c56464fac338eba1fbde0b5
[  527.185617] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.185861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.186168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.186255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.186368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.186626] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.186770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.186881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.187087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.187317] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.187462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.187572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.187767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.187897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.188048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.188288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.188428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.188536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.188743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.188887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.189095] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.189237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.189344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.189541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.189668] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  527.189887] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 69e928b0b51f44cfb5b8217f9052057f in queue reply_156db6c4256a4c58884afce15e6f5340
[  527.200607] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 69e928b0b51f44cfb5b8217f9052057f
[  527.405341] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.405341] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.405341] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.405341] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.405341] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.405341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.405341] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  527.405341] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 22198c7b05e64fd9b2a9a1e3dde78f68 in queue reply_156db6c4256a4c58884afce15e6f5340
[  527.416785] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 22198c7b05e64fd9b2a9a1e3dde78f68
[  527.536982] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-8b90bc12-de92-48aa-ab9c-83d0e44199ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.537236] env[60890]: Traceback (most recent call last):
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  527.537280] env[60890]:     timer()
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  527.537280] env[60890]:     cb(*args, **kw)
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  527.537280] env[60890]:     result = function(*args, **kwargs)
[  527.537280] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  527.537280] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  527.537280] env[60890]:     return method(*args, **kwargs)
[  527.537280] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  527.537280] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  527.537280] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  527.537280] env[60890]:     result = self.transport._send(
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  527.537280] env[60890]:     return self._driver.send(target, ctxt, message,
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  527.537280] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  527.537280] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  527.537280] env[60890]:     raise result
[  527.537895] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.537895] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.544048] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.544260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.544384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.544718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.544898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.545183] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.545340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.545467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.545745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.546020] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.546187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.546318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.546539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.546687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.546844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.547103] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.547267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.547382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.547599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.547736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.547998] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.550270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.550401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.550607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.550745] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  527.551317] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e4e8dba1764a4479be4e450428e534e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  527.559480] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e4e8dba1764a4479be4e450428e534e0
[  527.560647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.561097] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.561097] env[60890]: ERROR oslo_messaging.rpc.server 
[  527.692473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.692978] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.692978] env[60890]: ERROR oslo_messaging.rpc.server 
[  527.880128] env[60890]: INFO networking_nsxv3.common.synchronization [None req-c8d5de1c-98da-4dff-a17a-3e2f5b374e17 tempest-ServersWithSpecificFlavorTestJSON-1300140250 tempest-ServersWithSpecificFlavorTestJSON-1300140250-project-member] Enqueued Resource ID: a3a01724-483b-40cb-9132-f26a2a638504 with Priority: HIGHEST for action security_group_rules
[  527.880914] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a3a01724-483b-40cb-9132-f26a2a638504 with Priority: HIGHEST for action security_group_rules
[  527.881022] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.881076] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.881150] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.881332] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.881415] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.881563] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-a3a01724-483b-40cb-9132-f26a2a638504 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  527.881664] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  527.881706] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  527.881910] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  527.881985] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('a3a01724-483b-40cb-9132-f26a2a638504',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  527.882164] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0ae7a1ab69264e739265b5c46dde5b66 in queue reply_156db6c4256a4c58884afce15e6f5340
[  527.905926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.906430] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  527.906430] env[60890]: ERROR oslo_messaging.rpc.server 
[  527.938678] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0ae7a1ab69264e739265b5c46dde5b66
[  528.089864] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.090069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.090300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.090434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.090610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.090831] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.090976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.091088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.091291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.091526] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.091671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.091783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.091980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.092128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.092271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.092508] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.092692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.092808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.093008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.093133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.093361] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.093503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.093610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.093820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.093948] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.094171] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 74792b3557f9405184c3afb04a603242 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.099398] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 74792b3557f9405184c3afb04a603242
[  528.143403] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.143598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.143715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.143983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.144137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.144401] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.144549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.144732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.144961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.145206] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.145351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.145461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.145685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.145905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.146055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.146302] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.146440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.146548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.146747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.146871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.147139] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.147283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.147392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.147597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.147724] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.147957] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2785417eb55a44f9b86418c2beb03905 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.155113] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2785417eb55a44f9b86418c2beb03905
[  528.249186] env[60890]: INFO networking_nsxv3.common.synchronization [None req-cadebefc-fc18-404c-918d-d2f453bc1144 None None] Enqueued Resource ID: f5d24434-c513-4995-a05a-13cacb103978 with Priority: HIGHEST for action port
[  528.249439] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f5d24434-c513-4995-a05a-13cacb103978 with Priority: HIGHEST for action port
[  528.250025] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.250169] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.250227] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.250474] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.250539] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.250639] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-f5d24434-c513-4995-a05a-13cacb103978 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.250739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-f5d24434-c513-4995-a05a-13cacb103978" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.250795] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-f5d24434-c513-4995-a05a-13cacb103978" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.250997] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-f5d24434-c513-4995-a05a-13cacb103978" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.251080] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f5d24434-c513-4995-a05a-13cacb103978',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.251245] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ed86ac68e0ff43418dbd7bd6a4f230e6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.265451] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ed86ac68e0ff43418dbd7bd6a4f230e6
[  528.300606] env[60890]: INFO networking_nsxv3.common.synchronization [None req-e2d056c2-6e1d-4203-ba11-777dc1648f40 None None] Enqueued Resource ID: ef85f7c4-1716-44f6-8ba3-1fb1c6d34ef5 with Priority: HIGHEST for action port
[  528.301121] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ef85f7c4-1716-44f6-8ba3-1fb1c6d34ef5 with Priority: HIGHEST for action port
[  528.301442] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.301571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.301634] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.301836] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.301914] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.302062] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-ef85f7c4-1716-44f6-8ba3-1fb1c6d34ef5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.302153] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-ef85f7c4-1716-44f6-8ba3-1fb1c6d34ef5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.302213] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-ef85f7c4-1716-44f6-8ba3-1fb1c6d34ef5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.302411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-ef85f7c4-1716-44f6-8ba3-1fb1c6d34ef5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.302489] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ef85f7c4-1716-44f6-8ba3-1fb1c6d34ef5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.302668] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 494072b6bb384a5e87559f940f9341a0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.377004] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.377225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.377347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.377608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.377830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.378110] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-454 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.378255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.378367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.378584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.378819] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.378965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.379076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.379273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.379409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.379549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-454" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.379787] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.379932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.380096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.380311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.380438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.380677] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2aaf5c88-9f00-4082-bca9-53306b068c0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.380828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.380947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.381146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.381274] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2aaf5c88-9f00-4082-bca9-53306b068c0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.381503] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg be3feb7bc679419986bf2d89522534b8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.400697] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.400838] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.400907] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.401156] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.401197] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.401282] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: a3a01724-483b-40cb-9132-f26a2a638504 already deleted.
[  528.401362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.437399] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 494072b6bb384a5e87559f940f9341a0
[  528.439005] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg be3feb7bc679419986bf2d89522534b8
[  528.457440] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.457618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.457738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.458003] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.458139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.458395] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.458554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.458676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.458893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.459129] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.459277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.459394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.459594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.459728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.459865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.460129] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.460540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.460540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.460648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.460704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.460936] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.461078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.461188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.461390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.461516] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.461758] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5b74486ccc4d4f76a5b969ef2e271ab3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.489012] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5b74486ccc4d4f76a5b969ef2e271ab3
[  528.595713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.596257] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.596257] env[60890]: ERROR oslo_messaging.rpc.server 
[  528.625367] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.625568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.625700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.625975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.626130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.626407] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.626560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.626696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.626910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.627176] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.627338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.627452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.627660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.628895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.629103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.629365] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.629521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.629636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.629841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.629967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.630924] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.631089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.631272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.631408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.631537] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.631796] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f66f160e2f324022a46d9b80ed4990d4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.638103] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f66f160e2f324022a46d9b80ed4990d4
[  528.650794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.651088] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.651088] env[60890]: ERROR oslo_messaging.rpc.server 
[  528.690364] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.690364] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.690364] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.690364] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.690364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.696895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c4a6044c-b1cb-4e04-8687-e4446cb5275a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.696895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.696895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.696895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.696895] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4a6044c-b1cb-4e04-8687-e4446cb5275a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.696895] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6c3000828eab4dbb8183e59201499f04 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.703557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6c3000828eab4dbb8183e59201499f04
[  528.753610] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-f5d24434-c513-4995-a05a-13cacb103978" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.753869] env[60890]: Traceback (most recent call last):
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  528.753933] env[60890]:     timer()
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  528.753933] env[60890]:     cb(*args, **kw)
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  528.753933] env[60890]:     result = function(*args, **kwargs)
[  528.753933] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  528.753933] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  528.753933] env[60890]:     return method(*args, **kwargs)
[  528.753933] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  528.753933] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  528.753933] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  528.753933] env[60890]:     result = self.transport._send(
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  528.753933] env[60890]:     return self._driver.send(target, ctxt, message,
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  528.753933] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  528.753933] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  528.753933] env[60890]:     raise result
[  528.753933] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.753933] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.806517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-ef85f7c4-1716-44f6-8ba3-1fb1c6d34ef5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.806517] env[60890]: Traceback (most recent call last):
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  528.806517] env[60890]:     timer()
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  528.806517] env[60890]:     cb(*args, **kw)
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  528.806517] env[60890]:     result = function(*args, **kwargs)
[  528.806517] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  528.806517] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  528.806517] env[60890]:     return method(*args, **kwargs)
[  528.806517] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  528.806517] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  528.806517] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  528.806517] env[60890]:     result = self.transport._send(
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  528.806517] env[60890]:     return self._driver.send(target, ctxt, message,
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  528.806517] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  528.806517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  528.806517] env[60890]:     raise result
[  528.806517] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.806517] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.872818] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.873218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.873218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.873465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.873561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.873828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.873999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.874114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.874322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.874555] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.874721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.874836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.880325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.880484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.880661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.881070] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.881222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.881222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.884171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.884171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.884171] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  528.884171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  528.884171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  528.884171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  528.884171] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  528.884171] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 66dc65c1616344c8a7bed10bd1276b52 in queue reply_156db6c4256a4c58884afce15e6f5340
[  528.885348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2aaf5c88-9f00-4082-bca9-53306b068c0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.885814] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.885814] env[60890]: ERROR oslo_messaging.rpc.server 
[  528.893105] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 66dc65c1616344c8a7bed10bd1276b52
[  528.963671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.964181] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  528.964181] env[60890]: ERROR oslo_messaging.rpc.server 
[  529.076751] env[60890]: INFO networking_nsxv3.common.synchronization [None req-8949d211-6c1c-44b3-a60f-f1988ad8a53e tempest-ServersWithSpecificFlavorTestJSON-1300140250 tempest-ServersWithSpecificFlavorTestJSON-1300140250-project-member] Enqueued Resource ID: a3a01724-483b-40cb-9132-f26a2a638504 with Priority: HIGHEST for action security_group_rules
[  529.077288] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a3a01724-483b-40cb-9132-f26a2a638504 with Priority: HIGHEST for action security_group_rules
[  529.077615] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.077746] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.077814] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.078016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.078099] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.078251] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-a3a01724-483b-40cb-9132-f26a2a638504 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.078351] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.078408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.078555] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.078633] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('a3a01724-483b-40cb-9132-f26a2a638504',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  529.078811] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1f9d542716fe493c8ca5ab0ae6cdf52b in queue reply_156db6c4256a4c58884afce15e6f5340
[  529.095821] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1f9d542716fe493c8ca5ab0ae6cdf52b
[  529.134207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  529.134711] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  529.134711] env[60890]: ERROR oslo_messaging.rpc.server 
[  529.157365] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Address Group taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 971 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.157565] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.157617] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.157811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.166692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.173302] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Security Group (Rules) Is:1104.
[  529.173676] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Security Group (Rules) Are:{'6b38d532-e264-4d27-841f-0e430bb2d26a', '5a93c6de-4566-400e-b225-6fbc784def02', 'aa046b3f-6d44-4acb-8e58-50499e3f8d60', '839be8eb-d3ab-42d2-beae-bb4a6136fea9', '1efe66d8-51ee-4c77-bbc1-b0bff97bd6a0', '83621333-7805-450a-bf42-807dc0903fc0', '02fa8098-695f-4e4c-af31-9d2f20a9fc7c', '2c8665b5-7a16-454b-b09f-403fece4484c', '3c29ea98-dc38-490b-a8e3-abfdc118aa44', 'eb4ebc6c-f4d5-4c08-af40-def126bc447e', '38ad9e85-cb39-4189-9d61-20af96e8fe10', 'a84b8cf6-5370-462a-97e0-5c81e1e22633', 'aeb219c3-95b4-4de5-8fce-a41f8b49395d', '126abd34-fe92-4587-b136-35d78d6b2a21', '80e5b5da-5c33-415d-8683-524bd7b4691a', '2fdf09f9-efa8-4a14-a5c1-0e56ad631495', '01432ff8-955a-47c5-b665-9e3b030f909d', 'f8760856-2ea4-4f89-833e-7c672a2ef54f', 'f37d617f-64a1-4560-85e7-3656703fbd18', '214bfba7-572c-4d07-afe3-1421b94efaff', '26963d6f-35f6-46e6-8a33-94add9111f7b', 'ae07ce08-1e2b-447e-806a-b5f96013bf23', '001325e4-3b30-4168-927e-590103bce85a', 'e00ef03b-af7d-40d2-b5c4-f5bf948cc372', '9c9c140b-f706-4226-bbda-9950cb3e7e0e', '3b8642de-6fce-4721-aa49-9d1d63858bbd', 'f5fdf95e-aaff-43a4-8704-88046c16f0e2', '582e5156-771e-4f49-afec-56511f46df00', 'bedbbbab-f4e5-46f9-9732-8502ac4f5810', '94a8dda8-2e9c-4a69-97ab-ba0bfcc63981', '58c8aa49-9014-43f0-bf5b-d03cbade34c0', '577785e2-d134-40fe-b9a8-9eaf978fb67b', '0a023c7a-aab7-40a3-b867-63f307a7e3f5', '85d4d26a-9348-441b-a876-7f3546795321', '40faa2ad-a6a0-4e51-bc8c-a9fd92e740da', 'dcdd5ccd-96d7-43bb-94e7-a1099f4bfcd4', 'd7206195-0162-4ee9-a6b3-04ceb279bf3a', 'a880ae15-f8b4-4fdc-9f4c-4b0d5cb86453', '77128aab-c69a-4972-bea2-90cce0209793', '8b437b23-1e6f-49fc-a7b2-c532406ff181', '7662670f-bab5-4ef5-9237-479b1e61561d', '44d8f22d-5739-45af-a4ff-362aebd76359', 'd80d761a-2e72-4f5a-bb32-0e68160fbaf8', '3c210669-e296-4518-9563-be9d25f120da', 'ff43b22c-be14-4da5-b80d-36d8c54a860c', 'a30a9427-5e37-4471-9504-44c2d979c5b4', 'e72f919a-2482-420e-9b67-7355fec0316c', 'fcaf713b-1bcf-453c-8091-8e6f34635f0d', 'e014e8c4-efec-481b-9ed2-387c349047ee', 'a50769c7-88fa-4942-9e0e-4cb3595a5c30', 'a3c3f631-daba-43e1-8dec-871740520dc1', '78fd7d31-72e5-4a21-983e-ce48c7a7b713', '238101b5-c86b-4985-a776-3e973c0b60fa', '88520078-a760-425a-ab82-3525ef1c53eb', '92c9eef4-615d-48a3-84a3-cee217f08dce', '8837c3c4-5fbf-426c-a135-efc2f8c355a7', '37199e44-9bde-45c4-98fb-f8c75b0d3a5c', 'ba366f8f-c63c-4119-b420-2c303055cf86', 'f6c2252d-2268-4bb8-bc22-19eddc7610c7', 'aca54de8-c4f7-443e-aaca-7a111e51cc28', '2a81fc84-89bf-45e3-96d5-02a41988759f', '4a52564b-9d2d-469e-ab01-f09f3cdb4bec', '82de9bdb-0c15-45d4-9661-e40f8ec643f5', '13a8ff53-507a-470b-a2f2-fa671d340f4b', 'be36ad87-707f-4732-8a4b-162f66bfff59', '51a8e61b-e2a5-4064-9a56-68018ba339f2', '3b36b955-9c40-4a67-8695-6ebbe0cda689', '306a0b9c-142f-4a95-aa86-5296dbe02cc5', '8a7bf70f-b39d-4901-b42d-15428712deb8', 'a56f9f3d-b945-4159-bef4-429464666018', '6f226f67-b184-4a1c-9718-ac5c0b72e657', '1f67154b-3464-48f3-bfec-9852b358cbdf', 'feee80c5-5ba1-497f-bf8c-7b9bc27cfda0', '9e94d5b9-a9f9-438d-bf22-ab385101ba5b', 'bf9b416e-20fd-48ff-92a5-e238ddb61bd6', '10c57cd4-3944-4147-b429-b5b89c258f97', '670a4800-7ff6-4e55-9b4f-b46b5fb428a3', 'bbe619d9-4b8c-4abd-bc40-7b4017d42fee', '0d9bfa29-bfea-4478-8096-75c79e4dacd7', '0e948d3f-0f2d-4ff5-8578-e597302ee69f', 'e89dc789-cbe5-4c13-a583-6a1301d6acf7', '6e14057c-e840-4c19-9129-61edf2345a07', 'fb8ab3f7-dba9-492a-9091-63d448aff8f2', '0a4dd387-f79f-4f2d-a835-4a5e4fb9b5c2', '0dcb73a7-8912-48d0-a92c-49d62ca12e1d', '839c3392-f7df-4dac-99aa-faaa0ffbc1f2', 'd16d3b0e-751b-4387-9f22-804830b458a3', '1796c9a3-1cf8-4a66-9371-2d52508e5700', 'cbbc9657-8f2d-46bb-862c-6adbd8262def', '3bb43de2-ba3e-4750-9f47-debf358864be', 'ec9b93bf-6a59-4346-9df1-49b056a88ddc', '8b9ec5ff-f9d2-464f-aa26-034bfdedae22', '7011e504-95ac-4051-978d-74e49a6a417b', 'af98123a-50e1-4f30-848d-b652c92345a9', '7b3da12d-92d4-412c-a65e-9a5f565e9883', '14aedcd0-7413-4ff0-921d-0dead6637d4a', '68414d1d-514f-4108-8375-22274b3f2dca', '6cad175e-7e1b-470c-9598-263e629dba58', 'e266007b-24a8-4db4-a5fb-0b1cb7d27945', '18b4bffd-56b4-4aa9-b6a0-f69907fac144', 'ffbd48a8-331c-4efc-bcbc-83d55297b0c5', 'f2e485b8-428f-4e43-8b2c-236f117503d4', '2104385d-94a7-4ae7-8fb9-47feacce37d8', 'e1b59c88-2405-4b3a-9532-73440e2f5282', '37b44564-498b-442c-b259-5368844123d4', 'f2cf7c53-362f-4b7f-b49c-1c4e4bffc040', 'cc605c9b-1ace-499d-b8d8-fdb69ae3ceea', 'e497abb5-a45c-4bc8-b91b-ac72652a730b', '1f1db2e1-ec7e-4773-8ae6-022310685050', '2be9f8fb-fcb3-439a-802b-7c35d9fe987c', '4236e321-4341-469b-b076-101e05a1c634', '0fadcb76-b0b8-4881-b50d-7cc81fe937aa', '01b1ddc1-f902-4110-b9e7-1cb93ba198e3', 'b28c27bc-96c3-4d30-8b23-29cb54f22782', '6f0b1a90-314d-4230-9491-be3fd8160b48', '765a18e9-f395-4ddf-b33a-49588400d911', '0184a453-8434-420b-b0e7-af6dafd6be94', '6a14e5db-843c-4c73-a696-14fe0c37c9a6', '70195330-0f1b-46a5-b7d9-6b6bf9e4072b', '61737029-1c60-45dc-bf45-2b89048815d0', '8fb82bec-30c2-44df-bdf5-44d93a4ca5a5', '1550bb5e-b1a4-4e47-b433-714984a346f8', 'ea5b4a58-a322-479a-8f6f-40f420ac6961', '664e7649-5be7-4789-af43-679bf958c2c4', '0b100977-6974-4db9-ae62-ac6f9bde5f10', 'c6b86557-ee5f-4293-a23d-7962c990d0fb', '2804cc2c-4f0d-48d3-9112-48e6f6f774a4', 'ce0e684d-579e-4e3f-9712-705c0dcd4311', 'dc03a2e0-07d8-4b88-acd1-086899d59d81', 'b7705396-d1cc-4e43-b888-51eb0ddc7e58', '786ef165-7fde-4078-8740-fd9959125b95', '66387184-b7a8-4a37-be9b-a395c1435289', '4d89026d-4a87-43cd-89e4-854240fe7c6c', '41057d22-19a8-4ff7-ac1d-bc20e62be8cb', '4c8da2fb-5b24-4232-be92-e283831004b0', 'bb568b7c-9ed4-4a23-a24d-826aeae5bf41', 'e3654747-6ce5-4c34-9e69-227614e33008', 'a7c899e3-1e84-40d3-9d54-5c9ff71f8c66', '2e3533c7-7ed1-4a27-90f2-244125fcd07b', '2961c826-18bf-491d-91a2-2578410a18e5', '6f41e73c-277c-4be0-b93b-716a4236d8c3', 'e6fd6dd3-caf6-4e6f-8c22-0a6b73c984ff', 'ec10ba18-ca05-4a69-aac3-cb5e549941e4', 'a6f8758f-2516-448e-b4ef-6d330700c92b', 'fffa6845-9346-4dc4-8d2e-6da0881eac27', 'bb162c56-48da-4b1e-ac87-01508b852ed5', 'b6caa5d9-eccd-42f8-874a-8431edde4e92', 'f418017b-6e72-4702-8512-db7d24311693', 'd11614a3-711c-444d-9ddb-2c5adf8bdac3', '6f97d337-3e1c-475f-bd3b-7b17823178be', 'c22400fd-9ed7-4e1a-aa18-f1d92060ea9c', '0b55b0f0-df16-4b41-81d0-2ca9c5bebf24', 'a8261488-7d47-4511-87f1-4b70c625c70e', '55b71a8c-01f1-4e61-8b26-4ad3dae6dd79', 'fb4f905b-4a17-453c-b75e-b4ce53ef2d71', '4a1ff831-4b30-4354-bfb7-c0e0b77bf90c', 'e0d50e7c-efe8-4547-9e43-cb13e88bd4a6', '1947f963-faf1-407c-9995-b29c79427cb3', 'f380c991-18a3-45aa-9613-78b861f34dfe', 'f04d980b-f7d9-4cb3-9d9e-e4e7d419f9b4', '009cb6a7-916a-4da9-a2bf-c8d7a528d2f1', 'a6a45cee-bfda-4847-836d-5b6a15bb6d61', '1456471e-5a27-44c8-97d8-9cdb7d993ef6', '73ec4e34-3f1c-47b3-865e-c49be02a1450', '889f5e8e-b65a-4b4d-b007-72804b983a88', 'dccdd215-7b22-458d-9bc3-45907211a8d0', 'ca7c53f5-cf65-4e22-add6-b32f11f73781', 'd1234b73-ab53-4889-8eec-a220eb91044a', 'cc9e4024-2a52-40ee-b508-c42b42999bdd', 'a04c855f-2aa5-4c62-83d5-439effff24df', '04b45e82-4967-45f4-9139-e8c50f261183', 'd9c01ffd-4146-45cb-9ea3-3e3965976f33', '7be99172-ff42-4033-82c4-835b0a7cee87', 'cb1cefeb-b766-4610-8d3e-19e0c163381b', 'ab12b59c-62e4-4f5f-9fa4-4fab0bf511ee', '4ae77807-7394-4fa4-b332-6eea374322c3', 'a95dc000-d4c1-4bd7-bd1f-ff8845c97940', 'f0b34f41-c6bc-46e7-be6b-3d655bf3c2fa', '9f20533d-1fd8-4673-a050-284b825543ba', '59cd27d4-8ffc-42a1-bb7a-dc2c137b1933', 'ed9d63ef-ed9b-4000-9c9f-d97486bb6313', 'f198c82e-e1b7-4eca-aaf2-7c18868f4600', 'ce40d545-6550-4aeb-a7f8-a735b53b8141', 'af954bb5-a10f-4799-a3d0-10a0d55313bd', '577cd8aa-d141-4ac8-8b74-29ff70a034a0', 'b4e81915-13ab-4309-ada8-9791e009e78c', '9641406d-76e3-44cf-953d-d62bb3cc38ea', '3fad8bbb-41d0-4ba8-8a5b-2b0d23302f5a', '600ee48b-6826-48b5-aadf-b58abd34f5fb', '9291a66a-88d9-4f40-a93e-02b5b5d3a937', '464c0160-2f24-4146-b155-ef58aa98875d', '85306219-2678-44ae-8ee6-700b28ca0c16', '69dfed45-5682-4fa8-bf65-d94bf0a958de', 'eecea5f3-9ecf-44ba-a1fb-bb0e38a8bae5', 'b78ca953-47e4-4994-b97b-d5f30cb26160', '45da1fa0-516a-41f1-beaf-7b0047938b36', 'a0000fb0-481e-407e-9a95-a78fc3f44d25', 'd2c8d550-8a95-4b31-b00e-12fdb6ac3bc7', '4d6fe570-837e-4a57-8d3d-49be88b2101e', '3d623604-6f61-4d33-8501-7ad81f063617', 'c568d725-1788-4353-9dff-531c9bae7d67', '954fa2fd-1746-4b01-ad9e-aefde97c7f30', '01dbf841-f24b-4efb-ad32-65d63130f31b', 'fb5a9539-4230-48e4-9fce-b0cd6e6ab6d0', '63e33c7c-ad25-4466-9ea5-137a5a078ef4', 'fc596c57-8872-47e1-8e7f-e8f866d86176', 'be5b657d-31ee-4a84-9250-d007fe761bef', 'fbc889a9-3e3b-4f64-a82d-4a5e945ae91a', '1cc479c9-58ac-496b-b318-42eca2b4a176', '5b08f0e5-1871-47aa-b176-6c28e431cd64', '4e2e5f41-a617-4156-8637-a2b47a5a2bba', 'd1cacce0-33c1-460a-9c13-6c9bb82d5bb7', '1438c41c-3724-446e-9470-6b6d07775655', 'ee2b287e-1426-49c1-b950-eed64810790c', 'ee9399ac-b225-4d37-9bcb-ae77e14aa39d', '254d4d15-0a7b-4020-bc2f-72258c1c8eec', '1fdc786b-94fd-4481-9215-6709736862a1', 'af1f2fce-4e16-4cb6-8e10-37d3243d2e5c', '971a0755-80ec-4c36-bfd2-3d93101a5abd', '11dd23d4-bbf9-410f-bcbb-2b9b647cf24c', '3a0ee21b-ebf4-43dc-a5fb-920833d4cce1', '5c3fece4-9b5c-4a46-99e8-2a6e7a265c7a', '55860ec2-2462-400b-95bd-99a3222c063b', '1dacba0e-f104-44f4-9b0a-f9bd23c6b6a8', '58293dc1-5c7f-429b-ba74-e9d42d1666f3', 'a00cec11-d330-4a1e-ad66-91971ce5b9a6', 'e9c089e4-563a-427c-b932-4a216df08a57', '94a34b61-b616-4caa-9c26-c3155d20d919', 'd2afd0b0-c524-4918-969f-433a9ebacbd6', '143a19a4-7516-4d81-886e-da40672df71d', '785f1ec8-1ef3-403e-b97e-4a8a678a5939', '8afd7b42-456d-48e4-883e-5c3a5d379be2', '209e8f21-7a8a-4a2b-832f-6b32d1ec0ed9', 'd25ba1a7-6e37-46ed-86ad-4f4bfb7df314', '78b4e008-49cd-470c-ae8a-774ba969b7c3', '49e89fb7-3aba-428d-8f04-160ef9bcf002', 'a22e0506-a966-4a0b-a991-bb4a8408cff2', '30db5f24-5503-4749-b8e2-821dc53792e7', '6674dba3-bb9a-49ae-84c3-a2ddd00e8ad6', 'c0603c93-df1e-40bd-8ce9-6418a127d6b0', '5666f6ff-3d69-4235-994c-452c0fae18fd', 'c948f1fb-4350-4d5d-b1df-fb3a14ac4817', '3a3e9ade-6c09-4828-8407-26f3e688d5e5', 'c906dc90-d7ca-405d-a45b-815c94a71b7e', 'fecb9d13-77f6-4006-a1bc-bf7d840d9f5c', 'fb9757bf-6622-4822-8f32-038495aadc6c', '4896d82f-b449-4189-b263-fd11ab18490d', 'c84fcc4d-1a97-4cb2-8925-4874ee6137e5', '2cac0691-5a91-4653-8f6e-74a883d2af5d', '38111bd3-4034-41e6-9ccc-782c5ae900e7', '1ffbc453-234f-4598-abeb-445d743b2d73', '589f9f17-d562-405d-807a-d06ca5f19245', 'bc7a86f5-a363-4d65-a3d1-5cf5eeb5513c', '08bdd2f9-f2af-40e5-92c1-55503fd797de', '68ff0746-0354-4f85-bd0e-e18806b5264b', '09ebdcdf-8b9b-4517-a588-00e15d641284', 'b965b829-c919-45d7-8f57-bf91f7115ff4', 'e16906cf-be6a-4cd2-a595-03325aa3f6ac', 'd6b8dcbb-e14f-4f11-bb76-f75033b893f8', '9f12fd63-092b-4cf3-8856-28beeef3485e', 'db061e8f-981d-4327-af00-5df74d7c5d30', '5ffed754-cb53-48b6-b90e-7a84d20cbcff', 'e1836ab1-010a-4a7d-b4e4-17caa2e57c69', 'e9d3bb15-9510-4ecb-9944-7b670002f3db', '4aca9e0d-b421-4d5c-97b8-1ff1c05076ac', '8ac9c40e-78a9-4ada-ae66-4da570b5cd9e', '24153d44-a4e5-4967-970c-02cc1409cfc4', 'a2c2c5b1-61d6-480e-987c-36b99a885833', 'faa09b75-06b1-4c1e-923e-770d142e3387', 'fedc0335-bfdb-431c-9ebe-2d0c5dcca610', 'e817d23f-de35-451c-8fd2-ca9c50526fc0', 'a0e22a54-72bd-41e3-9f5a-2fb084d17ddb', '6a3f10a2-a6b1-4d76-934c-dd426e139ba0', '56dbb27f-1ce7-404f-8a52-a85fdcc16099', '42659b80-961d-4ac8-8dd1-c8abdd91e151', 'ffdaaf16-0c6b-4a84-8f94-ec37d83fbf93', '8a00232b-280a-49aa-96cc-4db769877bdd', '770d3355-bdad-40a7-9165-8c87037ccff9', '43da1381-ceee-4b61-9744-379c9888eebd', 'f73e1b31-a070-4800-9257-1b25323936c5', 'c44bf0b3-e403-433b-90da-1f246c6f80f2', 'b1aa9247-ff1b-4b54-8d12-128f806a3983', '69ff7038-e436-4385-9ace-76489f5274a4', '9c9b2b37-25f3-4ea4-9705-88a83120332f', 'fe26cac7-fdcc-4383-a30c-d8de8a4d3c21', '16bf5aeb-22b7-4088-8df3-2ec0f278a90c', '60b683d5-e141-4352-93cc-20bcdd311f74', 'c424aa8b-f99b-43d9-8517-385e52405670', '35a53a62-2092-428a-af42-c36aa4a2e4e1', '1d5b1c86-5db9-4624-8257-59f524e8ce7b', 'ccf33da8-9d54-45f0-b267-2705cf6a4186', '7dc11649-1c7d-4afc-ae36-afbc6a0eb062', 'e0fa8e7f-6095-4051-971f-3d56e8b47ee8', 'a43d51e2-fcce-4c08-944b-cf9ce5083ab7', 'e3450d94-5f06-487c-8f1c-328bc6fba19e', 'b1797cf8-cb1f-4ebb-ac16-6ed10106b9ff', '3af8c6cd-f2d4-4c35-8e15-38a6e8156123', 'dcf5b496-a31a-4753-968e-77837002ca21', 'fff09ab7-441d-4a0f-ae90-674a718ec4f3', 'a4f9530d-abf9-4fd5-b392-1c74596d99d7', 'fa8397df-4f59-44b5-9c06-5d5820e0bd7a', '964275dd-587b-4d13-a15c-5e2e765dc173', '8238829c-bcc9-47ae-8b4a-f56da8d9bc6b', '662725bd-a5e5-456f-b838-563b326e3a69', 'f7cb0d08-2713-4610-ba93-1d22dd5b1e0a', '8f3dd00b-3728-4286-be1f-f6049b5ede30', 'd36e049d-d9b8-4376-97f5-c845ff70dedd', '2c2b4845-5027-4121-b64e-4d71a1abc374', 'cd57e4da-789f-4a64-bbf7-ce1d90673fc9', '5b92549a-ed5c-49bd-b865-19eaa2482b09', 'b6bd256b-05fb-4b80-8c50-7ec02c917143', '346f57b0-657d-4301-97c8-44b7290e6430', 'faf7e41c-e9f5-404b-9221-0c6b012ee740', '3d87c3d1-4f69-4fcb-8b9d-da1f031cdf07', 'cec17d99-408e-4595-9216-0339c1b1a12f', 'c4bcdb1b-9690-4b93-a9b8-55e2e3a3ea9b', '8121b060-9737-4559-b17f-75f227ac9048', 'c4de4049-5c54-449f-94db-e624dc87ed63', '7a5788cc-73ae-44da-b85e-b25ef06234e3', '3a74b557-b525-4dc0-a0e7-e14d50c91d0a', 'cdb1148a-23a6-4b57-b907-03ffc91195a8', '823e793f-a8d3-40fe-be20-df3b5ad9179b', '5aa069f2-bf6c-4c7c-8fec-a00897397e56', '1cf9d498-da4a-4180-94d2-0d5c00afe650', '6aac90cc-9579-460d-b2ed-04a81c109de8', 'e544e2a3-735d-4503-afc2-47d009f7935f', '7b08d4dd-a0b4-43cc-9d2a-972b135c1430', 'c64b8f84-967c-4071-b291-34ba48437ccf', 'fbaf7512-2908-4177-8ed2-42a865d610d6', '62753820-42fb-40da-b36e-a401f377357a', '6a3758ec-79fc-48ca-aac0-e30d12d3b6d9', '3ca70d72-b4f9-41e4-b92c-8c029e1cd2d7', 'a65dbb38-6e75-41df-ba67-0e189e721981', 'ce9c7edd-74b1-4fc3-b808-b611a0c61df5', 'f7b396d3-bdfa-4346-b07f-e6b751236aad', 'f3778041-429e-4499-9e3a-0a0ec8f16936', '165d33c2-263b-401d-91df-0117a29be480', '68fd8c46-2475-4e55-a9b3-42ee20771cdc', '945fb4ab-2407-4ec6-95ab-d9be92246a1e', '6c4fda67-9ddc-4459-afa0-be683844dd0e', '25c8a813-432c-476f-aeaf-b73c7efc2337', '711a3536-173b-45d0-9b9a-4d98244ffc01', 'ce1fd1a6-558d-4261-bc5a-b9b1d3568adf', 'f8132790-8261-4615-82bc-c992a0dc4d0c', '92026d5c-2af4-4b91-9583-fc592f5f71c0', '1bf44a11-d803-436f-87d8-597ce52acd11', '388fd340-e59b-4d1c-9db4-11b4ca6a731b', '609deb9b-c3df-44cf-a011-5b0ae760bf55', '7f558621-b3d8-43f9-b03e-954ad5175cec', '9e35f44e-6f91-4f66-8b91-436d0a57466e', '97e79043-346f-442f-94a1-fe8e792245a1', 'a44385e7-cb47-4f8d-9530-d060cdbb30db', '3dd464bf-f81b-4f06-91f9-9cb3c015b4a3', '1284ecfe-4f71-4dff-a589-8ae7cec46a06', '4649c51c-cca0-4694-973a-c9be70500ecd', 'eac9fec0-2641-4426-8f8b-3a48a0b32821', 'ad07e91e-1c49-48ce-9ae1-335ee19c8982', '1446656e-6f64-47a3-8a54-95b85571d145', 'f1bf0c6b-53a3-4503-811c-e63935f6cedb', '623f3cf6-106a-4a41-b707-92839c6859e1', '17f1a49c-29d8-4874-b0ef-5dc9a69026b1', '73f26647-937f-4b0e-baf7-5f7d891ab022', '6566d6a4-8755-4145-9c53-c66b367852fd', 'acfd2e4a-e422-4863-b861-bd4c1cbfab87', '27eca136-22f1-4f3b-a5f4-47f2e3f5ef28', 'ba02ed24-3ea0-4d76-ad58-055b1ca60908', '1502208c-631a-4cc1-bd71-30d4da929264', 'd72bc675-a968-4981-ae8d-4b8104a8fc94', 'a41600f1-eaf3-467d-bda1-3490acc6c6ee', 'b6829360-ebaa-4356-b51b-2856c130a04b', '95e4fb59-1a70-4dc6-b2e1-3a1b5cd6f0cb', '5e85b3c2-afd2-486a-8324-3da82026e742', '584726a4-ee3c-4cfd-873b-5c09940aace4', '4da78a20-e58e-44ef-90b3-719d970e3608', '5452a771-253f-4a3e-a7d6-3878af982983', '8be25d2e-54f2-45ad-aec4-f0c41d0c6960', '8e6b865b-b83a-4ec0-9137-268a6bbf2d46', 'eed75b04-4032-4491-8700-12abb6192242', 'e18fb7e2-a107-49e1-8559-c8a453a0c12c', 'f1e12046-5ff0-48da-978b-5054e910d3d5', '4e7b5bb8-5dc9-44ba-bd86-01b2fcca4f35', 'e065b96f-c7a7-4109-9a98-0abc08279f41', '6fc1a1f7-48c0-466d-a902-be41bb2909dc', 'c3558061-f684-4797-b74c-55e8de4a383d', '279e5a6f-86ae-47dc-934a-ae06083151a2', '84b96d48-bc75-4aaf-8339-ae9a2bae6798', 'efc90b59-f1dd-4eea-99f9-3b9a58b88483', 'd140e637-4054-4d2f-9545-669a57df1ea8', '3db62012-d8bc-4017-ba31-98fb2b592936', '3e024ad4-7bc8-4446-a722-e846d3e48e0b', 'ede7815e-f2b4-4e92-a8d4-e7f530e83c70', 'f3ef8423-bbec-4546-8d54-c4d2d9eefb3d', 'c1939d81-71d8-402a-a72d-eed70e81ec98', '1af34663-8243-424b-b749-35aaff17c778', '9a21c4b4-f54d-4459-96c3-c9e6ae9e8e31', 'd421d774-ddc0-4979-a5a6-cef76679e898', 'a053d09b-356c-4582-8595-542852913eaf', 'bfc6b91c-e131-47a7-81fa-88e5eb3c5fff', '241e9854-bde0-4e7c-be1c-dea37a2f0e90', '4ca6f1e9-0c4e-474e-a1cb-d41125407f0d', 'd4253539-4006-4dc4-a613-30ebad87735b', '40da80a4-7fc6-4866-a0e7-cc1154336ea6', 'e2c554bb-3bf6-479a-983b-5e58478ab704', 'b8416495-ecf8-4e60-816a-0db10f2c5d2e', '1230ed50-7883-4176-9792-c60e73edc163', '2c269fc8-c778-4161-8be5-10533e66ac9e', 'c647a34a-44bf-41fe-a35f-1481ecc31b22', '0bcc9b1e-9da3-455a-a32e-4c1324c91bd7', '09e4d63a-c555-45b0-8879-05ed1216b496', '77b2202a-fd3e-412c-8aa7-ffc8b0cf8f50', 'fdae3cbf-c8b9-47ad-b362-9e1cdcc23b87', '4af1a18a-6733-468e-8cac-856f966abb42', 'c0996d40-8bae-4a49-965c-513c1adb118c', 'c20809a9-47ae-4379-ac6e-d799c88889da', '45a5677c-96a7-4a37-89d6-9b7035f0ab14', 'fa9bcefb-95e5-4595-89e6-77d7b9e49f82', '9ad76aeb-e965-4d2a-a9de-92f04fb9b8e4', '79f4e8e4-d10b-411c-b47f-7969eec84600', '7597ca1e-0b70-4a71-bb82-922c1a9b52a2', '608073a3-82f6-4577-b38b-71ede8712317', 'f42f230f-d6ce-464e-8025-66a23d253f1a', '02da5604-ce37-4d80-b7c0-dad352952562', '8829a113-df8c-4cc5-8530-5b9d1f318fd8', '6af90a39-eb33-40d8-a4a4-a656b94df580', 'c639d0f5-bba5-4c41-bbc6-c92ce0a01206', '332bb205-cae2-4814-85b5-1225b0d42faa', '757c7c0d-a608-4026-b7b2-8764639b83c7', '9bfa128c-a182-43b5-b5e4-ccccbe0be779', 'fe982f5e-8999-46cc-8ae2-5c0610479a91', 'c908b30c-f7bc-48e7-9b07-5f652939eb79', '289b8cb5-827b-4b96-b41c-146d1d612207', 'f00d8ffa-e5ec-4126-8232-b3e64acef8a1', '7a58b996-fabf-44f8-abb4-3ec0d3dbfeaf', '49e061d2-9378-4f1b-9816-71de3f03f01a', '29571673-755e-4d4c-9728-1abff3f8fd67', '6c0987df-8e1e-411c-b19d-adf095a1050f', 'adb1a1ad-e570-4fdf-9488-538bde8d32f5', 'ab612c9d-05e2-4ab1-a2d4-9146bd0bb6d4', 'c4a645d6-4eb7-4201-99ae-fe3faa482744', '1e051928-63f4-4795-9722-b10db743f215', '37dc3b34-d2d1-4a5a-8b3c-eb401e782bdb', 'bc493fef-5f41-4377-8a24-001b00f280e7', '0725ccf8-beed-4f7f-b741-fc54ff7e2ebd', 'f2f1f0e6-1821-4054-bdef-4f3c76348178', 'e911664c-510f-4c67-8ff6-d67968070e84', '218da72c-3ddd-423e-86f7-d0aed4bd1e47', '5e33dc80-b973-4d0d-ab5a-c6a45198254b', 'ca441aa2-6796-4db8-960b-a7977e434d43', '58a6fe6d-8bef-4419-95cd-9fc604bdc421', '8365d17b-d86d-43e0-852e-f3fda8eae2d8', 'e594aff5-132c-4d9a-952e-1fc74d243a12', '958c0b6a-6f8a-4f6c-8406-f556f75af78a', 'e40a313f-2d65-4606-90be-d168ff24a3f7', '4d21ad8f-4e3b-49a4-96fe-8f66364a0d0c', '1abbf587-b255-4721-b8f0-2b96dc32a646', '1185d64d-2d10-4e81-b958-ee93bab94ae5', '7cd32ef2-1ef1-4f42-84b4-1cdcb23ab30a', '56de7b18-ad03-4d6d-bfb4-aea2c90a6d47', '08f8156c-4637-4906-b6b2-fcaec641b75a', 'a3da1b8c-c49b-466e-b18d-613f8ddc3b05', '34a104f7-6762-4110-ae85-4f192a1db9a5', '92779f0d-dd69-4016-beee-33e8705804a9', '1f9b0305-e137-447b-b185-53a69184efd6', 'b49295ac-1d35-4294-bba8-6aef3b287be6', 'fba18575-c2d9-41e3-83ba-ec8fcf23c984', '79fe113b-9305-40b6-91b2-c9f6abd84439', 'd3ad96e0-1d90-49e8-af43-e3d38e6f7128', 'dbaec471-51bf-4ec1-9822-7a92e9f1b77e', '93dae16f-4011-4ff2-a6d1-c013cf6dbd77', '2c8fc7c8-df5a-4914-b0d7-83b71000b835', '5e83a431-918f-463a-966c-6b402e6021cb', '1be0f48f-0825-45a4-9b3a-913bf013e327', '6838c4a5-f996-4a37-b8a2-15e2a0a4ffcb', '2f8ba902-14bb-4b79-a17f-aeff5c95c84a', '8ecfd421-bd58-45b9-b323-979d6e3630b0', '580aa329-04e3-45e2-8acb-fe14cb26013f', '04e49400-b1fd-43a1-a29b-b2b7deffb80d', '131fa278-c4ed-4af0-b2f1-1fb805dc2a51', 'f8530d7d-3d78-4c4d-a62c-e627e9d28065', '87e1bac4-6dfd-4dd2-ba6c-bf8a90c5453a', '4a26f80f-e99b-40a7-af80-ddd13ab9d582', '6fffeb0e-5192-4855-b76c-325a13629478', 'b65e9824-88c2-4a62-aaef-e317a555dff3', '1244147d-e91f-4a43-9903-ec672032953a', 'a3e59377-ee7a-4002-a95b-02dee37cae6f', 'e261ceb3-7a43-41a8-b870-105307106d6c', 'dbea1d53-4616-40a5-b389-56118fc689c3', '0ecd2cdd-7394-4aae-935f-083440f44c73', 'f830a09d-6403-46af-b959-8c9068eccf11', '85e6e031-95fc-4ce2-9737-b9b457a11759', 'e42b080f-17cf-42de-9fde-5a75f5d6da7f', '28e519e8-50db-407f-afc0-b7bb0f37ecbb', '066a337d-30d8-4291-a38c-3c98191037df', 'f29fc06c-9bea-4105-8982-01646af59e1e', '28dd8e74-e4d3-49fe-9918-84cc4ec0f499', 'ee3929a9-b431-4bc9-9334-fd309a38f152', 'd664efb9-22b5-4dc3-b65c-0b8ac6b52c2e', '3398388f-e461-4c4f-8673-88cd40aee29e', '358a54e2-b53b-44b4-98a4-d301e8b0d87c', '583533dc-a2d6-44dc-a848-e70aad3c5ee8', '11623846-afc1-4b5d-a7a8-70cf65f99d71', 'b174a6c2-747b-41d1-b7fe-88d2ef3b94fb', '2f50427d-088e-47a2-a79f-c9f02938e184', 'f4b13771-8208-4490-9e07-10bb00f4ec2d', '74960b3c-125c-469d-b324-99edb8902762', 'a0ff327a-4777-4ea5-a473-953e99f48fce', '1d6fe346-cbba-4f0e-ad78-ce674e51562e', '21882b2c-8f85-4025-a05e-9d5ac5c7dfe5', '8676b965-8b56-49ab-b6a9-a54292a4c53f', '022cc406-b4f2-420a-b707-82b789638d65', '712394e2-8cde-4fc1-828f-1ea228fb85a6', '3f0f56ed-0942-4f28-b2ab-e015705a388d', '1ff1ff9e-a535-450b-b5e5-2b3aad2a92e1', 'ee0feec7-0fc2-4f57-8eea-f3d9e2999db0', '11782be5-5a4f-4d91-a1f7-66a85f90ce4f', 'd3681b6e-accf-4e7a-8599-9732dff3b868', '9809779f-51b0-45e9-b693-edca5cfa39d2', 'b9b299b3-6398-452e-9651-6189a49e5fea', '010843b8-b1a4-4ece-95e1-d8430a677f49', '7300c61a-eec4-49e5-a60b-1ac573d1c2ca', '3e8314ac-6de4-4fe0-b3b8-8090e7238d54', '53437029-9e9d-4417-8d26-d440ce43ab8c', 'd3a2479e-d6f6-4769-ab79-f8f399872ac2', '3dc7a690-4000-4337-9052-3c175d5b17d1', '984cf644-5297-4ea0-8e48-171a2b9c84c1', 'ed944c6d-d921-486d-96e8-2d616bba357c', 'fca489be-e88c-4540-b01f-3eaa4cdc376e', '777fabc7-034e-42c9-a21b-bd8346081baf', '2e646167-2df2-497e-a3a3-3c3ed4e65936', 'a68a8b48-b54e-4fda-b0fe-f939c87763ad', 'dfc6871c-16b2-426b-8d7e-f339a06594ac', '63047547-b550-47c9-b993-2e886acbf560', '5812319e-e240-4a68-ae1c-eff464ba0a7e', '7e185c0b-5d07-4465-b049-bd18fc2048c9', 'acc316ea-6132-40a7-bdaa-c82630a45736', '9faf5a46-083c-4779-a318-0a209d1bfff9', '1c25d563-9932-41d3-acba-b0ca5aefe8c0', '12afa242-3b02-471b-92ae-01a7a666d975', 'd27cb9f0-260e-457e-811f-ffd1448009cc', 'e638a2b9-395d-46aa-9f48-ad893802cb22', 'e16d1536-3624-47ea-a0b2-4be9049977d5', 'c9ef6859-d2a4-4586-8477-efafa7d2937f', '048381bd-f60c-40c8-85c9-867a04ffe5b5', 'd944bff4-eef4-466d-abc4-a0323e8486f7', '6baa86ac-ebc5-4808-94cb-30af7cca84ac', 'fa71937f-fc79-464b-ac33-3f84913bc5c1', '79cba46b-3a56-43d4-a711-ed1c27f8a8ac', '2ef579a7-8663-40f0-aef4-ba3069611485', '310f4a54-46a9-43e9-b8e2-df926bc1fc13', 'f61b7271-7b94-43b8-b404-a9037f54d120', 'ccd2e48d-222e-499a-8e5f-ab3d417b96c8', '5377676b-7e17-43de-9646-f00a238b562d', 'c824be34-b6bc-4635-aebb-4714244319bf', '3b813add-6d07-48bf-b3f3-300ea3fb0c85', 'd790a0e2-32f6-48b7-9c6e-dee360e54606', 'b6f88864-38f3-4ce4-843d-1807f88ec790', '56f5050a-0fee-4e03-a7fb-334fc44234f6', '186a9c80-281c-4fda-acd9-271fafd649ad', '8eab6bde-c670-45bd-b91e-094355c25bb7', 'c3abbce1-9d00-40a5-9d38-2aff1aed5129', '9bf93fdd-2fad-4c46-a7a3-896d816da7ac', '28d55ac5-c96d-4867-970a-5a69a0bfa928', 'e4d7cf20-4a9b-46e3-a36b-4dd1d1ce97da', '6dc0566e-ead8-43a5-be56-dec682c7c63f', 'f96e429c-b74c-411b-87f9-eba9d50e53d1', '82879c65-a05d-448b-b0f9-6a7083ed4590', 'cdd34ae5-533a-49a0-ae1b-2dcab7087683', '2effa546-1ce6-449c-b3d7-0564561ec4ac', '8fd62893-b488-41fd-96c5-61a7ff5af1bd', '440ee2f9-d226-4f4f-bde1-d14200833b29', '011229d4-5e90-407c-a670-49ffc98f79d1', '5cb60160-7c7d-4b59-9203-2c5ab07842f1', '65a29211-e00c-439d-a26b-bbb119eb9630', '4c8ba24f-6d56-4d7b-91d9-68aae2ae8f74', 'd3181ea2-2e2e-4151-82a0-6dd3bf9aabba', '1bd6158a-c5e7-47ce-b0f5-6ffdb35d17de', '8ad9e821-9357-4562-aaf1-7bfd3b93fb20', 'ead39029-9ec0-40ce-9bad-b260809b8ced', 'f77386bf-9902-4005-93b1-73c0f9f397bb', 'cde94ddc-3f20-4c89-8a3c-9bbb54d26a99', '78758cbd-9265-478d-92d9-b7cccd2f4751', '8687f805-b0a3-4260-8d2e-3c87b697ce0c', '99a5ca37-83e4-4e3a-9187-587f8a2caf20', '6f9ba684-3830-44cc-8ed9-29e1389d71be', '9dd5a52a-3035-4902-a140-1b44b469fb5c', '697e286d-b7ad-4447-9b92-81fcd59c3236', '60d17e33-b6ec-4501-8063-2b3951911a42', '8af0aa51-01ab-4f51-82c2-9d4649f04066', 'c2b5a36b-d306-4db1-a57a-bfc5066833ff', 'ace9ff04-b28a-44ee-9eca-c9822f577fcd', '047eea06-4eb4-41bf-8874-8bea4d8e056b', 'd7227627-de83-4167-b4e9-1360bb1c99a8', '761e0a99-e835-47b3-adde-5cfcc0d6e48b', 'fbcc04fa-12f2-4242-bdea-9c53ad092f13', 'ce4707a7-28e8-41c7-a022-0d4a7b2a02e8', '1f945598-5c9e-4cb4-802a-74d8682d8cd2', 'ee83cf79-3a92-4dc3-8a01-38c8d2817290', '17deaf2b-ca98-4873-bf41-35e0b79bcddd', '022dee22-57bc-48a6-aec2-307aee481dc8', 'a338029c-77c2-48cb-aaf9-d6708dfcc628', '3b9eb7b6-bc65-4476-bde0-75f07d1a479c', 'cb646236-e11c-4a30-a4a6-b40305062d2c', 'b7d15d9c-e206-4355-bc66-600e5727a2db', 'dc274eba-cdfa-493a-8b22-6e418157eca8', '04ce3382-304c-4c10-b79b-a288e40525d6', '9fcbfee9-0f98-447c-9669-0bc475f8554f', '9df3dcd2-7333-4940-abc6-2902bec425b6', '74965ae7-000a-4b65-8ae0-28821722a4c5', 'c0ea3df6-66d6-494c-af93-599cc8005459', 'a099661f-0f96-4b74-baed-ee0ab9870f16', '37f6587f-2a6a-4021-b5c9-e38c3842e331', '83ba575d-c051-48d8-ab84-0e7a4a0397a8', '3da14946-2583-495e-bc5a-eef658d15c95', '6a51f7e6-7131-46a5-8fd2-a4347e699df6', '4b3860e4-68b3-4a9d-bb0f-c79499310593', '5333c492-671a-4354-9d2f-7d73251f1c39', 'ea7a55e5-3749-40aa-84e0-9738398e69ae', '4adec779-4085-43f9-9749-a4f79f5519eb', 'bdf82a38-e0e8-4f61-a3a9-ab71dd33fec3', 'c2fabf99-bf05-4453-8f83-cc8eca721e65', 'c8670329-b9f2-4472-95b3-14552103e125', '4e7ba94f-e9c2-4941-b6ce-383d82c0f65b', '189500e5-532b-48d1-88cc-579b774e8f7f', '07029ca3-e0b5-4d9d-81c7-22db124dba19', '5dc731b6-a1b1-458f-be71-b41cca193acf', '7d9a6280-47ee-4957-8405-1dcc209613bb', 'd97a3c4c-df2d-4051-8c51-b3078efc48d5', '5715f1ac-2a29-4b6c-97d3-8fa5ae66c115', 'f7d16528-a3c8-4c12-a34b-1b98d3e2b218', '4d45ca7a-94ec-4d4e-8f32-20a567f835e1', 'a9d36583-5e55-4934-9f8e-d8d22497ad27', 'ca83cf8c-07f7-4246-9428-991ce4845a38', '2d918405-702e-48ab-bc40-3dc1da4a7a2e', '86a210bd-92f9-414a-a467-2a842cfadaa9', '80eff49b-b490-4df3-a771-4b0bdae7592b', 'cf4ed4b2-dbba-40f2-abe6-19802aadf541', 'eba3010a-a026-4e9e-94a8-ae0f77724cb1', 'a8068f5c-6f96-4bff-b79d-a3dcfeeaa280', '0abdc94f-1499-4384-b38e-c8823b487f21', '928bfa94-b605-4714-b124-5a25c5e42bd8', '7ed3d189-3a99-4816-a1c7-5678c119d8ba', 'cb30f59e-dd28-4a75-94a0-259942493875', 'dff956e7-f33b-4c03-ae57-9538dd622b91', '8bf15a9a-69cf-4b50-b60a-514b36ae96df', '8f174427-d4fb-4f4f-a59f-5e0345bdadd9', 'a17c90cf-7203-4d92-b33e-0f3831ffe006', 'fc182a83-5472-4e2a-a9dc-7e0012e483c5', 'c7afb8e0-c173-4c1f-ae90-a1c65093017b', '41613cc1-c095-4291-869a-df02557c715c', '7e7eb2c0-d1f3-4347-827b-0acfce995056', '84ab8378-05a9-4ffc-af1b-e77135596962', '733e3537-1785-4484-b0cc-33df13195ad2', '59f24c01-a820-46f9-9ec5-26ccc2bb9b99', '22a79425-10fb-432e-a949-01a37c22c495', 'd40ca857-6493-444f-92a6-52947467ad7c', '02d1f251-1551-4f4b-8ace-5a89a897fba3', '97aae706-e442-4648-a616-eb9daf306a4f', '9969d379-361d-4a47-a876-150d83e0b7ba', '802ebbfe-ccd9-458a-93a4-48509d998992', '5eeb1e61-0ea5-4f40-b98a-4169bde6fa30', '124df5fb-bccf-4daa-847f-7576261e09eb', 'c5e60296-9c4e-4eb9-9df2-26a6d27ac31b', '597170dc-c5e5-40e2-9686-a29391e2f08b', 'fe1c3779-aa50-4fd7-9794-23f1b10f447e', '3851d527-deb9-4e67-9ca9-46a76985faeb', '0902ee22-e88c-4f0e-85b9-5e83fe2f0b30', '4efb2bd4-7b01-43ab-8d5f-b80aac7a71a7', '19ae0fce-1d97-4bd9-95fa-aec4f8cebc52', '6d90d74e-28ed-4911-95e8-51ac37233027', '49e33507-c0b0-4bd7-8209-f6ce0dbf15d0', '0dbe9eea-5849-421d-a500-6cd545a2c12f', 'ec2ed287-28c1-48dd-893b-c8cda0a508b4', 'd51bc6f2-1e50-4673-8e60-eae257787176', 'b5cc1352-4268-4513-a03c-bf633544e2dd', '264dbc8a-7fa1-4220-8d5d-c008824f5870', 'abac75c2-3ed7-47af-8df4-2c1597b67964', '5fe05a90-e94c-4c88-a16f-d82799a9bef0', 'fb50169e-785d-4403-9853-adffc6b5171e', 'e70c7461-4be5-437d-b243-abf069ac3ae6', '6f561a81-b15f-43ae-9cc5-eeb0f9a38a71', '1503fb5a-3fe8-4bf3-9eb2-7a903286756d', '0f3d6f25-b150-4f89-8f19-47e05112e968', '7ee44e78-1c9c-45ca-9757-b59f1531492e', '6aeada42-ec65-4bdf-8831-113036a29ffa', '49cebfb6-383f-4c28-9b4e-9a8c875ca489', '3d817822-730f-4917-9401-22a2a406fe8e', '7d054de9-59a2-449a-83f0-aae1c12fd104', '2b1d43d1-9bc8-4fe7-9f1a-04f6c98097ff', 'a44e1a27-2f77-4160-825f-63e89fc2e3a4', '29d3dc08-b4fd-4b56-815b-5f9fa7d351e4', '7fcfac3d-00eb-4bb6-a342-5833b1f3037c', '02a71b1b-d156-46fd-b8e0-919d3e68ab67', 'e2d02080-838f-441f-ae03-5b4e77cde9c1', 'f7c506e4-0ce6-4792-89cc-3ffd47246bec', '69e19bec-8ad6-406b-80cf-51ab5d80b65f', '37ba87ed-b988-42ab-b71f-19a26582f05f', 'eca7eac3-64d5-4db8-aac1-7d2171b5b594', 'e010414f-d677-4efe-99da-b7b66755d4ca', '6bfb0168-5032-40ec-a757-5f9ceeee0c65', 'e2d76a70-0ae1-40b6-a610-899a66e92716', 'ec03565e-b463-4894-a441-f6eaa1e14497', '1fe06423-042e-413e-aefb-31ea7173c770', '7c4d266b-9ceb-4335-943f-3decf874437f', '09c24c3a-991e-4e6c-86dd-00775ddc9db8', '1a1f374c-636c-4ffd-bba9-795be4d7c2f2', '663a0264-041d-4bed-b616-6e20aeb74b39', 'f43951b3-e42d-409a-aa7e-19dff1102007', '9d22157f-08d5-4c08-9fab-28d2e2ae7b57', 'c6ee3756-e8bb-44c9-b0bb-119e9f03b360', '34bf7959-f4bb-401d-b934-7920d274a088', 'a7616d94-b3fc-42e2-98b2-03bd04c97d0f', 'd312af10-76f9-43c3-9118-db3f81bde31d', '48c74bc3-3e15-4ba7-a5ac-786074cab253', '3e87029f-6dd3-4cbf-8d6e-a4f83bc8bc19', '9c49fa14-3e11-4b12-af33-d1398946ffc3', '9985ed29-19b2-4ca2-a114-d4a5db55a349', 'cf2c4e64-dad1-46be-befd-f8dcc859abcd', '8d96ae9f-9dff-4863-abf3-283f80f1027e', 'ee3322a3-875e-4848-9aa9-bfe069f0ae65', '9e62b556-144d-4381-b9f4-c58077f08685', 'e1b38e4c-4612-4d4e-9489-9953af45bea0', 'e191de29-8260-484b-a6a8-4d5ac2d0faca', 'f41d25da-3599-422e-bc46-f1b0d4c38eba', 'bbf86abc-e599-4bae-b673-973dd4c77365', 'e782a415-6fb6-40c7-b28d-d91e58ed44c5', 'b20561db-0067-42ad-8721-9326a247ffcb', '4481d451-0060-4cd0-87ed-b9a00dee1243', '7f811a3f-c3c9-4981-aecb-6bee8144bc66', '141d3859-76fd-4cb7-b64a-8e89bb6105e2', 'ad19bfc9-a815-4cce-a18f-1e50273a4940', '29ba3db3-4c1c-48c2-9785-aa587880a1d7', 'd534f8cf-207f-4b9c-b3e7-63e5396c976b', '91066cd2-7197-4c51-9c8e-3ac6a7f8685f', '55ae8638-cc22-424d-bc8e-304f641175e1', '21ddf401-722a-4898-b34f-e3dbc81ef122', 'aaf28ac3-74a6-4b67-9d27-2fb8b180f9e3', '5768f84d-bc45-479d-825a-5974920907ee', '1d3ba8be-748f-4508-83ad-056e89ecd39f', '70e9491a-e894-439c-b190-469564028fee', 'fe5e2980-4176-4c9a-8e8b-9cd47c9f7e07', '2d528d3a-e365-4f26-8393-65491081b279', 'bfaa69dd-d009-4377-be48-167890cd0179', '1ecf7494-7210-436d-b926-9e873e5b6529', '0ae7bd49-a7c4-4a09-8011-f70d483760f4', '95cf7921-2ddd-4b0a-b393-82942167c148', '8220e6e0-1274-4067-9362-d1843efa59d4', '2664d12b-7727-4ba7-bb20-6d36d52f12f0', '70edd117-06a0-4f49-bf35-29f4c7de3064', 'ccf74d3d-94c4-4ec0-8f39-eed826c566f5', 'ef765d30-9529-409a-927d-f25515bf1710', 'b3b126c8-2a60-4802-bbaf-1b098e51ecce', 'd015e83c-3cad-4f5e-bd30-4dbe071befaf', '8986e901-9e9b-4ed2-b31d-f2176a202a62', 'e0e50c73-fc45-4794-99e6-aa137290a4e1', '7572b39c-6969-487d-be36-ca0a69fdcb8e', 'af434786-c3ee-46f6-8d10-acc5ec0bec29', 'ba575443-b4fb-48f1-a008-c3aea0628271', '5dfa1861-20e7-4866-b924-cdf621dd0a88', '548d42ba-3381-4893-a39f-2237d2752a23', '9c6e8871-295d-461f-911e-2a16aa881389', 'b4b49aea-8d72-4ee3-81d5-398dd7085be2', 'eb27e4a2-d332-4a29-abcc-6ca01d089d06', 'f4f4208f-01e3-4f3a-9135-4574a328dcee', 'fd1cddde-992f-488c-b63e-eafcae157326', '9e561b7e-ec42-4d1d-9695-d93010948fa2', 'b5b4a96c-c1b6-497e-8191-a4b3f0bc39be', '51b22c8f-efd3-4377-8141-3b16d0476e5d', 'b25ae027-ca8b-4a81-bf78-94ec4c0d66ab', '34b97a40-57f8-4326-bcea-73dd6f2309a0', '7a3f1db2-3f97-4d58-b359-59fb7cd9ca49', '48df3a64-b065-4d84-ad45-92911f48d13e', '8b648bb5-7a71-47ce-811c-2049fdc0273e', '0676c759-6a02-4f46-916f-8bd3c99563c3', 'fe74f926-5f68-454e-a893-149be82f3346', '457f3c6f-f4ec-4749-97f9-117dfc06d4db', '5a728d3c-d769-4afd-853d-9d2552d239a5', '86a1e722-f662-43fc-bd17-037bd6123f06', 'ce3871ad-3efb-4785-a306-e49923889613', 'd1f00b2a-1df6-4873-b29e-3aa53818d384', '1322bff5-1e46-471b-a79b-e5da27677247', '1185d178-8e44-47bd-b4dd-a296e2bf9031', '3d4e3921-e785-454f-8ff1-f625205a8808', '23896746-3201-469b-bdd7-a0c2d3f396c7', 'c2e5ea10-1af1-4814-9cd8-ec569c0fb0de', '447eb45f-4d31-4eb9-98a3-cb5c05fbe1e8', '2e4464aa-97ee-4116-8c9d-b9c99123e0a1', 'e874d62e-9f2c-47d0-b5bf-84a9765b48e5', '1bebebdf-5387-4a03-b616-d4766e9e0630', '1b49a31b-1828-4674-9db4-1823b2c9d607', '31dd60d6-8b0e-4abc-81a3-e7bf5aee1cb6', 'f94f9ada-e246-472f-8fea-233c1b8bbd47', '24a7e24e-8902-4872-acc4-aa3665594333', 'fb4837ae-9d8f-4e78-8039-45d62225e224', 'e0add8c2-9f3d-4557-8317-3fd19c913636', 'd1459ed9-6cbf-4387-ad59-972de8282692', '5c026cf8-d0c0-4784-8058-3a035144b3da', '2d157506-234d-4988-8bd7-88e7c35d19b0', '47420f4c-9e29-401d-a29a-1341d1127262', 'd43ee8ea-6cca-4f7f-9bce-a3a8e5e7dab4', '2645464c-d1cb-4d10-9075-17315791f00a', 'bb97eb22-25db-4f75-88a5-abf0eb3c8af6', '35bf7f9c-a62d-41f5-a3ba-99402d1f1eab', 'c4f807a4-41b6-480e-b973-86ff7e73ab21', '22138115-c9b8-4e58-aa35-367f13b91e17', '92de6bd1-222a-4fb5-bb41-120db189a0f7', 'd46b57f1-3a75-4338-8808-65582f8d94c8', '9f584829-b266-48fd-98cf-e86a4235fcbe', '17aab31e-3103-49c0-b8d6-9759bbe79a84', 'c14c1e16-eae8-4d5b-9ddf-b72d64c43125', '1492808e-c3c2-45e4-8518-cab2612e22e7', '66caa0a9-74a8-4cce-89cc-2312d253c6f7', '5a8460ef-028b-44c6-a4bd-d59dcb0796f7', 'ed20d25b-c10d-4f19-a527-ba3986779d48', '415de96b-e6bb-494e-b35e-966c7dd7a87f', '84fe4c94-9dd2-4322-9fbf-1016ee227291', '27fe07e1-d4f3-44bc-a492-2f9d1ddeb0c7', '302ea960-1230-4bf8-99fe-27b1f9067cc2', '894364b3-fb27-4540-a326-1f5d61b78745', '1684acf9-ef02-4e79-a090-bd3e2a427f05', '023631b0-dcd4-45ee-b15c-814f0cf38a3b', 'd6093c90-218a-4dfa-b851-e6683be1b50f', 'c7a0ff84-0f87-41e5-a8da-5257d8f22fbc', 'b6b65099-73c3-4abb-bd1c-842f26e18531', 'ba4e3148-fcf9-42fb-a0b8-8ab4b52ee982', '1a5ed077-8c88-47e4-ae7e-2008eb4c3e22', 'b6071397-8be8-4827-885a-45bbc6d1fb4c', '9feb48b1-9a04-4ba0-bc24-f63ecedfe23c', '3c6cb31b-9f2b-49df-8b06-50468e85fbf8', '31622995-c2ae-494f-b685-3bc348ebd21d', 'c08ae808-8493-4daa-ab35-daa380ee9df8', '80f8c2fa-c4c9-431c-9ae8-23007b27a85c', '34a888c2-3361-4ab5-a980-e3bfa6efc30d', '0904f757-7efb-44d1-a07c-24ff0833a5bf', '8c4347f0-f777-4bda-921c-83dedf80e1de', 'c92ee54a-0b8e-4a1d-abdc-ae1f5fc7826c', '96153093-3cfe-4742-904d-082fda5ae357', '08b7f452-154f-40dd-b211-38c0ab23a43b', '9c891958-ab84-4fde-a1ca-5d78eeaa394f', '1418516c-8077-40a6-9f51-41b9918afc9f', '024d6120-f86f-4e13-8155-9b20e55d99ad', 'c87fb18d-b128-44be-9e86-8ce394233d49', '34b1ff62-2372-473f-b93f-cca96e2eb61a', '56fe2f0a-c17c-4d2a-93ee-b06d2fbd3dff', 'd38d30d2-4da0-4138-97f9-211255fc288b', 'ab228ff2-1100-4100-ab63-88ea99a7e943', '506dd0b4-fee5-4dcc-874e-f0235bfa2828', '26685da3-b9c4-4ed8-a558-2ca25be64085', '7322b9d3-3a26-4da4-a750-af006850174b', '0514918f-5ffa-4e77-b220-8b64830a40a7', 'bf43b8cb-967c-4523-97f3-011be9762178', '7b5d0d66-16ad-4291-8b06-099a356cea4b', 'd5eddf04-7beb-4a20-a045-33d11424d8cc', '90a0a1ba-c8c2-4e00-ab60-42b4d59cc516', '4b414d7a-cbf0-4624-88de-c805138d9469', '6c173c55-d1c9-4285-9b10-e21d678e7999', 'a241e0cd-2994-4d43-93b2-100bf77f7da7', 'c6422c29-1ec6-4b3c-a7dc-d909e4b6c776', 'e17fa03f-00c2-4084-a0f3-87bf51645714', '5809b5c8-d3fb-4995-bbe9-46b35ba7a837', '501088ff-9114-47f3-a7e7-36457bd82fab', 'f936b0ec-fd29-495c-80b9-96023b5fa460', 'd08b5426-3ef3-46b2-ae28-910d7b7937ca', 'f01c43e6-1957-441b-898f-b2c6273ba646', '3b128da7-919a-4a50-a440-f685220695ab', '8818059e-66b1-4ac0-b638-d427ce75bfc4', 'dde81f60-5a15-44f3-b60c-c7b169e554a3', '16a1b6cc-dced-4ab1-b9be-0b664667a67f', '8819cc44-6594-44a9-a97a-6df141cf952d', '040e807f-1b40-4a81-a096-ea383cdec0ed', '56b7a25d-b800-4320-b4f2-aa1668ecf1da', 'd206b9af-4e0c-4a6f-899f-020da44952d7', '53cb9724-e752-4070-bb8e-da16f0be0ba8', '019aa88e-c52c-497c-a823-a89b306889b1', 'f68874ca-c5fd-498e-902c-d872a7d16ee8', '5d2a5b1f-1089-4250-8aaa-f525a55043d6', '04cbcd12-88b1-4650-a755-8ad8499df957', 'dfff59bb-507a-4c4e-b9fa-2cf08b448a44', 'b1995568-c79d-401a-ba73-390016f8a02b', 'a905c316-7ee3-41d2-9b55-6e12c3317b9a', '29a2382d-fddd-49d1-98ee-d0f129b4994f', 'c9b92b8a-b5d3-4d7a-9229-8a450f5ab16a', 'ce5c0cab-9e87-4057-b6df-4045faf73d59', 'af3b6cf3-10d3-4277-8d01-2f4a9d448120', '96dfd8dd-9c48-4fc9-b886-fcbc74b3a6aa', '5ec36386-f181-4891-a12b-6c37896b6bbd', '371b2cc3-9ef5-420f-923b-b6c32920bac0', '80b20778-c6c4-45d4-8710-e55b22534600', '17f91db9-6a80-49d6-be28-b95abf1b8dbe', '4a9b7c39-7855-4c17-80ab-6a3f09c2eff1', 'd835f022-ee2a-43af-999e-cfeccc326ad0', '9cbe8973-fcb1-48b2-944d-35e1183ccb08', 'e4ef434d-8c32-4783-8bfa-4b355dbede8c', '0a44e499-75e2-497c-9343-f1b4262c8ed5', '791c6bdd-fbd5-4b2d-9d9e-7c2c3d0be271', 'b6e46e39-96f2-4857-8b8c-05946f97398f', '68be3455-0d3d-4252-a25d-e98d1f93959c', 'db22c6ea-fbf0-4591-a740-fdb11f104cd6', '5436bb08-3024-4498-8189-60954bb6c170', '0750c8db-f160-4546-bc94-fb7a61cd5001', '6c07253d-cead-49a8-946f-7fe0844137bc', '54855400-8642-44e4-81cc-7075cfea6f37', 'ac26a1e1-5de8-40c3-84b2-3f604cd7e391', 'fae87739-ac61-4ea0-9923-f2bd1b7bcbb5', '2c0509a5-6ec8-4390-ae2d-02ba73bab59e', '2f006589-53f7-411d-8279-359c69fc2b1f', '6f788244-8f9b-46dd-b5e1-b19f3a5e6131', 'c24041ef-b239-40de-b347-df426e0a2073', 'c7e71994-63b8-4fe3-a6ef-f962925aa112', 'c2d5a1a4-760c-4554-9d08-9992f08d35af', 'f78ecdfd-cb09-4f8d-b5e7-6e1d6adf8f88', '1c2563c0-56d0-4848-b2c8-d85691baa0c2', '549c7520-0777-479c-9090-8c83d7ff44fe', '5b423e62-762a-4ab4-b45d-e3081843e829', '4c1cfa51-39f4-4782-8e18-e32bcdaaa296', 'e79a895e-c380-485d-ab80-5072fdd94906', '0cf1c80e-7b02-4407-b59d-9a87f051217b', '77cf495c-34d0-4bca-9533-835c7062484c', '50ba7074-a1f3-4c47-bdca-ebed33745690', '3776d9dd-4495-42e6-96f1-0900b1a651bc', 'd0351fa3-86b9-4e5f-b602-5cc26b2b4e1a', 'ba7a5c70-ef14-4223-940e-d9b65a5c2f44', '0eb15e18-7b3c-4baf-b6ba-49cce461f02d', 'ddea4258-9527-4c8e-aa23-04c9f39d454b', '74088f01-1e96-469c-94e0-f61dc139ab51', '0afd8c89-7ad5-49d8-a85e-39ed10b37b11', '9787f01c-8a0d-4df6-a587-5d30a565f15b', 'fe2952b9-e54c-4fcf-a3f4-0d1b0cdcf532', '128f391d-c159-49cf-a223-c29d146e605c', '2d422992-adc4-4918-8695-1be3ea54fad0', '5443a9e4-61b3-4a58-b622-e0865e818197', '95cb1ba4-475a-4fba-af27-6e299b8693d6', '78a13011-7ede-43d7-9926-d26a847a0367', 'a944dbaa-8f75-4eff-8e48-3600be37c913', '9ba225a5-a1cc-4ebd-bed6-c5b65c881038', 'e0444955-bb69-4d70-9137-acb2a70d80de', 'a1aff3ec-86a5-44d0-b936-91b5f6019078', '2ddf1d36-57da-4007-8272-96c8e3962564', '304386a2-c613-4f16-9327-3f6a2167058b', 'ea28d9ae-a068-4a1d-b763-7020fc76a49a', '9cb0355f-2e29-4d16-876c-0458a685a829', '52be54e4-79c9-469a-807b-0fb6d455b788', '64fc0b3b-d89c-47a4-baae-fc6d5cca946d', '43dfd9d5-6650-4815-8a95-9e55a0457528', 'd736675f-7033-400a-92c7-757bf805b112', '393cafae-c886-463a-8e0e-5138959e1a67', '51883c97-7a92-461f-a1e7-6d0f3ce88999', 'ef917ef5-ddbb-4398-828b-4467007c4b32', '030ac600-cb66-4641-8f22-af782a925ac0', '2a52bd60-903c-4fbe-97ab-4079853853b4', '847d3905-a68a-47cf-9569-f9ff452358dd', 'e631b00c-caa4-40a0-a619-1938e43365ed', '15ee5af7-6e29-4af8-ba31-65da47953658', 'c29771b5-48ee-4d68-9980-793abaf98b19', 'c10f7bd8-9d31-4d23-b0a9-f340ff3aa1d0', 'a49ae307-917e-4aa5-92ed-478f0bccc1d5', '74b6746c-d008-49ce-b752-583ff964da8c', '234a6ada-1f19-4af0-b30b-2715e9d6827c', '22a33e3b-2807-4070-b4c9-548afecb263a', '4d0530d8-d9dc-4311-b635-8903c20f89cb', 'f52103bb-f7e3-4164-a161-ef984538455a', 'bfbd37dc-5a95-4ffe-a366-16c9b9bf557d', '55ec0987-804b-499d-9d94-5fa8c1f94bb7', '204a52cf-b02c-43eb-a468-09c0d3a8bf5e', 'f840c68d-b8d4-411a-add9-79ddb4a5a01d', 'b0535bfe-3132-4a86-92f5-e737403415f1', '8be7bad4-d871-4e0b-b739-c83dba0a6b48', 'ee97aa03-f966-4850-b6fa-245ad3f06746', '52aa95e0-2ad7-461d-a359-3d6f0ba3bf00', '1ff650aa-2505-4f9f-afdc-62affd4a59f3', '491e29c6-4036-4672-9499-7b8d52381b75', '9139c65b-fffd-48de-9b9a-6f3e8ca3b58f', '3c66e192-a190-4d74-b725-52b496ae4e71', 'b397eafa-4896-45c8-88b7-296ab4c72f90', '4cb9b6e3-3826-4e84-8379-1f90b822e929', '90e9e729-d7f6-4641-ae0b-248da1d41c1b', '57f7700e-fa43-4490-afe8-06a649b1edef', 'eee3c5f8-5463-466a-8f2b-962983fda9d6', '6045b6c3-2ae7-499f-99a6-831538329df6', 'c9d6a978-b2f2-47ba-a226-5fba26d893dd', '89b2bca8-bf9e-4af0-9013-e7fdbd1ca74b', '52ee2513-9fcf-44a7-aec9-05cc0408de42', 'b21a3332-ba3c-4abd-ad81-710b1622e68e', '47e84f03-09ae-430d-9abb-d071de9c8f6f', '108d4049-e88b-475e-a248-afeaae2782ef', '5dc7ce26-be22-44be-b7f7-d7195e3d2a88', 'dda8d9e0-ecd1-43bd-aee7-9a9bd16ec428', '4bd1fd10-9ba1-436c-8752-27b7a4560ddb', '4095fe81-5265-4056-bf0d-9dc5f205f721', 'c19bee4f-1b7f-419f-a258-3024a1e24362', '2b1643a6-dfa2-4737-8167-5d2f44345f62', 'c6664015-b8d1-45c3-9e0f-30ed1fcbaace', 'a80b5b94-9818-4a0b-827e-ba7595f24f7d', '7350bb95-48a2-4e12-84e2-5adc4a27aeb1', 'ddc5f6f6-9c3c-4f88-8f6a-0226adcb5919', 'ba30fa04-e616-49f3-9d46-24e1d872b9a4', '3f9ff912-1e91-4605-9249-c40022e202d7', 'd2c30556-8bc6-4dd4-b2fe-c4c3c4ab5885', '6d3d901a-bdc5-47cd-8883-6b4effd4ff7c', 'e3932a7d-dfe7-44ae-bc26-c40afd5f96bf', 'a39488f8-1cda-4fef-92a3-0b397ee5f847', '696c2f1c-a5a2-4245-813e-3c11aa0c33bc', '4045f62d-ccb1-4d3d-bda4-1d85573774ae', 'ca6d2373-81b7-44ac-9383-0a7b44218b4a', '478c3c4a-8272-4ae3-b40c-3768035edbae', '40633562-4688-40d3-96ca-174c7e57eff2', 'ba1e7fed-7979-4496-9b2b-a296c1905013', 'f293b8f8-d4b6-42f2-bdbc-c596e6b8c9d0', '6e34871f-b153-4397-9f96-f967f99a340a', 'a4a9a24c-9bf6-4b4c-b27d-7928040a6be1', 'ab940563-ed69-49c3-9432-9dd0242c38dc', 'b704f1a5-a183-440d-9620-3a5eb025e872', '8760b3e3-dd7e-47c4-84f7-e6ffc9ed2e28', 'b1913a3c-939a-4179-8747-8894039bb9d2', 'b8bddd94-c115-48e8-836e-d2c0dd86b4c1', 'c81493e0-1680-4dc1-9ea9-342b2c6a2c3d', '120da6e4-646a-4e93-8c48-633ecb6b5463', '5b5148bf-3080-497e-bd0f-ee4e0f043ce7', '92fffecd-d047-4272-93f0-e34fe1095654', '1d12cf0b-ce2c-4a35-ba08-b3c207d5e833', 'b23e5e4f-2045-4010-a8cf-b08f78bb8a7f', '3576ff00-12fe-467d-b0b3-70c17e6fa7fb', 'c60b0ae9-3caa-4f4f-9fea-bc87ec18db01', 'fdd72e16-c00d-4f61-9064-fdab83067032', '7830c781-83c6-4a39-ab31-12e93e49971c', 'ef33333f-13b6-49b1-a93a-7e1814c1579c', '1dfe49de-acd2-4751-a309-010e4c80746e', 'd2667211-fce8-46e3-bb93-9fb6f96e2862', '2ec55b7b-20b7-4cf9-b1af-7eebc566c5fd', 'd29ed45d-0ac5-495e-b8f8-0a3f5bca1d68', 'a64c69a2-30d1-43d1-8168-276cdccf9e9a', '20067308-9771-4a1c-98cf-0897c25af809', '43d796a8-4572-4d85-9070-c0e2e2a12e83', 'fb3ef62d-77fc-4235-a38a-1cb6a8f5988b', '7a09516b-1eb1-497e-a6c2-2cbe33bcdaf1', 'a9dca19f-c4e5-423f-b3dd-42b299f03a05', '3775bd14-c4a6-451c-8b65-a80427853797', '100a2224-3dec-4029-b3b2-83910acd126d', '3878a9bc-f6d7-420e-b7cc-1c01a5b88a7d', '1f99ff9c-e148-4911-a841-72fc13a6715c', 'e7627f2d-71ea-47af-9726-506d71cb7008', 'b4a105c3-b28f-4ff4-bbf0-d077055c4047', 'b2064e5c-3e87-4d5b-bf26-f72f6a7f3057', '8bffddb8-9f0d-4702-97f8-260774038c37', 'f99db900-76dd-4e80-ad56-1b5ba62ed0b6', 'e4ceda65-fe1e-4e04-bc5b-095ef9d287ab', 'a06b53a5-5d79-453f-acee-6acc4233380d', 'b3bdb06f-cbba-45ca-b220-9ff15c0a70b3', '9e17b192-16d3-4e67-94d9-f60c52b202b2', 'd7216c28-bc23-4e22-8070-7ca66d10fbaa', '2abb3181-da0b-43ab-84b4-b05f9a71006c', '0783acfc-6542-4e0d-95dc-688beee0abaa', 'adb284a2-bad5-4764-b598-4ab9d292e393', '4e56ac00-e29a-425d-97dd-76a0eca4185c', '731a9eaf-948e-49c0-b104-291038854e3d', '1fc924be-e382-4bd0-b044-98cb6337cacf', 'd1051d18-d9e9-4cfb-9d34-f68b4f116570', '14ddd434-dce1-4c75-a4b6-ab9074d0eab2', '50ec2195-9903-46fa-b266-69167a128a5b', '200bbbcf-9a01-41b5-8b06-954e88759a42', 'ab8310f9-d92e-42e4-83d5-f04b8a826418', '6c9ebcca-bff1-479e-bb26-217cad961da4', '544d33c7-2d7b-4ceb-8cc6-bbfacada9ba9', 'df0a6bfc-a117-4ec8-b2b8-7da7eccd33d8', '31086c8b-ca13-444e-b1aa-e37ecde17fab', '82fccaa9-2976-405b-bc44-444cff9c2815', '6dff6735-c2c5-46b0-8a41-69167bf2fdf7', '5dcdcebb-f642-4dce-a91e-c5a5cc7295bc', '34c6808c-3d1b-4940-bad9-194e18819726', '914b7b3d-7c0f-4f3b-a1d9-38f22996809c', 'b4d9941c-dbf9-4f20-a7fe-bc18cdd88f46', 'ec479475-c53c-4241-8214-59f5cbb0cba0', '2e52f6c4-8141-45a1-96bc-b013a4f4250a', '3ea47f07-37cd-4213-8b55-0987189f3fb5', '98f8dbec-5823-4e7b-b1da-4a0a66dd1841', '6ac1b426-5073-4f20-a6d7-1223585831f1', '3a68d0db-c832-4dfa-9990-900d2ae4052e', 'dc24ea82-98be-49f1-abd8-30a387a82c69', '7b2d625e-8cc2-42fb-9e29-b34aa426c58c', '505644f1-8bcf-443c-910e-56529d51d2ea', '615c020d-21f5-4546-a5ee-fd44d03d85fc', '1d58f4e4-9bea-4bdd-b2c8-cd8f0d9c453b', '63513359-8a62-411b-befc-832b186e26cd', '7d06e6b0-c3f6-4ae1-8ca9-b8caa6855a5d', '85cdf949-7be1-4b5f-92e1-e3727eb8fe94', '387d0ce1-8293-4dce-9768-a1c209033753', 'a0ad096f-db7f-4482-8cc2-7351d09f0e73', 'fce2e2ed-a8d1-4aec-adcf-7994c0652e5b', '3f6b4f1f-7647-4366-a686-c9e305a630c2', '30851f37-ca48-46ec-9eef-a58f074d08ef', '8a0c460a-3834-4fc6-a8d6-e9c8543d36ab', '28b73b0f-d5b1-4281-8525-9fb24f4b217c', '0aea1091-b42a-4c8a-a204-27c189cc5ed8', '6bd99aa1-0b1d-415d-af9c-94f28f1a0042', '919e9c5d-58f7-4390-a195-3a9f9a5171ed', '42600244-505c-4e82-be7b-426daa23b7fd', '9fc4c493-901a-4947-860e-43db9656684b', 'f61d1915-5424-4fb8-9846-ad37acfd3909', '1f2318ea-b550-4abf-b342-a78c067ce3af', '4695169d-6637-40db-b9c3-a14133da2bad', '55f38b46-4ad2-41c6-9723-a684ad69e88d', '456a3a88-691e-4539-9ecf-0f80d149e8c8', '43dec340-cc37-4cf6-8ed3-e1d0f1c61fc4', 'dbcbb0d3-532c-4074-9392-4551fc760b23', 'a04a033f-037a-434a-bd51-e902c27d7cf4'} {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  529.174851] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Segment QoS.
[  529.192300] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Segment QoS taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 122 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.192455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.192606] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.192817] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.192925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.193030] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Segment QoS Is:0.
[  529.193102] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Segment QoS Are:set() {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  529.193185] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Members).
[  529.199056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c4a6044c-b1cb-4e04-8687-e4446cb5275a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  529.199537] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  529.199537] env[60890]: ERROR oslo_messaging.rpc.server 
[  529.364783] env[60890]: INFO networking_nsxv3.common.synchronization [req-35a4c6f8-720e-429c-8df5-13f67850aad2 req-87215a94-2198-4550-8e41-94afe3f85669 tempest-VolumesAdminNegativeTest-2086354477 tempest-VolumesAdminNegativeTest-2086354477-project-member] Enqueued Resource ID: b371c264-efdf-44ab-9a75-9ff23ee9391e with Priority: HIGHEST for action security_group_members
[  529.364783] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b371c264-efdf-44ab-9a75-9ff23ee9391e with Priority: HIGHEST for action security_group_members
[  529.365190] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.365288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.365288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.365459] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.365546] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.365758] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b371c264-efdf-44ab-9a75-9ff23ee9391e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.365883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.365942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.366112] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.366247] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.366336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.366390] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.366515] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.366600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.366751] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b371c264-efdf-44ab-9a75-9ff23ee9391e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  529.366863] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 90d7af7a0e32412386faa56747e0cbe3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  529.376353] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 90d7af7a0e32412386faa56747e0cbe3
[  529.387306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  529.387306] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  529.387306] env[60890]: ERROR oslo_messaging.rpc.server 
[  529.597365] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.597661] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.597661] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.597814] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.597907] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.598005] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: a3a01724-483b-40cb-9132-f26a2a638504 already deleted.
[  529.598085] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.765789] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.765964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.766077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.766347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.766492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.766749] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.766903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.767024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.767238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.767475] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.767625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.767746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.767951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.768104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.768246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.768489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.768636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.768751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.768955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.769097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.769309] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.769455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.769567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.769771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.769912] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  529.770133] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2a87a5707d41450a87bb58fd2a2eb8b9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  529.773167] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.773337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.773458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.773678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.773805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.774043] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.774194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.774308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.774509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.776218] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.776218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.776218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.776218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.776218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.776218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.776530] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.776832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.776967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.777231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.777379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.777808] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.778081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.778215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.778478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.778622] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  529.779030] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8f9939a2b60d46e8b986619a797d1a47 in queue reply_156db6c4256a4c58884afce15e6f5340
[  529.780455] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2a87a5707d41450a87bb58fd2a2eb8b9
[  529.783512] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8f9939a2b60d46e8b986619a797d1a47
[  529.869074] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.869237] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.869303] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.869529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.869628] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.869813] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  529.869923] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  529.869982] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  529.870125] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  529.870204] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  529.870290] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b371c264-efdf-44ab-9a75-9ff23ee9391e already deleted.
[  529.870371] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.156404] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.156594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.156712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.156977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.157110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.157386] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.157540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.157663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.157870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.158102] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.158307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.158354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.158560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.158724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.158872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.159112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.159255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.159375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.159571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.159697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.159944] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4c0f3b32-53c4-487c-aead-6ab30e2c256d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.160161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.160278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.160479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.160604] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c0f3b32-53c4-487c-aead-6ab30e2c256d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  530.160833] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fda7fca1284144e1833e82520e331a90 in queue reply_156db6c4256a4c58884afce15e6f5340
[  530.174721] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fda7fca1284144e1833e82520e331a90
[  530.281092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  530.281659] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  530.281659] env[60890]: ERROR oslo_messaging.rpc.server 
[  530.282896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  530.283310] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  530.283310] env[60890]: ERROR oslo_messaging.rpc.server 
[  530.377863] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.378110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.378237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.378507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.378649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.378914] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.379366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.379366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.379448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.379630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.379778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.379888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.380221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.380363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.380508] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.380877] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.380951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.381573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.381817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.381993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.382283] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.382401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.382519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.382728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.382864] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  530.383094] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 729d8de8c45f4d06908d3589f0e027e9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  530.388174] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 729d8de8c45f4d06908d3589f0e027e9
[  530.662656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4c0f3b32-53c4-487c-aead-6ab30e2c256d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  530.663195] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  530.663195] env[60890]: ERROR oslo_messaging.rpc.server 
[  530.740846] env[60890]: INFO networking_nsxv3.common.synchronization [None req-c40f5d04-d586-41ff-96b1-4ccf61aafe47 None None] Enqueued Resource ID: a744ac9f-9a7e-47a8-b2a0-0d86311039f7 with Priority: HIGHEST for action port
[  530.741111] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a744ac9f-9a7e-47a8-b2a0-0d86311039f7 with Priority: HIGHEST for action port
[  530.741694] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.741834] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.741904] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.742110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.742204] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.742349] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-a744ac9f-9a7e-47a8-b2a0-0d86311039f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.742443] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-a744ac9f-9a7e-47a8-b2a0-0d86311039f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.742502] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-a744ac9f-9a7e-47a8-b2a0-0d86311039f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.742845] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-a744ac9f-9a7e-47a8-b2a0-0d86311039f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.742845] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a744ac9f-9a7e-47a8-b2a0-0d86311039f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  530.742986] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2291d0c42ef04e4c930aceff6dc391ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  530.750510] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2291d0c42ef04e4c930aceff6dc391ad
[  530.929108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  530.929108] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  530.929108] env[60890]: ERROR oslo_messaging.rpc.server 
[  530.930950] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.931146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.931248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.931509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.931653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.933606] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.933606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.933606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.933606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.933606] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.933606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.933606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.933606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.934050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.934050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.934154] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.934225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.934347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.934566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.934729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.935304] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.935496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.935658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.935798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.935931] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  530.936183] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 30d8919980164a4b8cf789d71b5a2514 in queue reply_156db6c4256a4c58884afce15e6f5340
[  530.942647] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 30d8919980164a4b8cf789d71b5a2514
[  530.996417] env[60890]: INFO networking_nsxv3.common.synchronization [req-50675964-e257-4b9a-8b5e-255db72b5f0e req-fe0fc8c6-4935-490a-b434-664670189ef6 tempest-DeleteServersAdminTestJSON-570518421 tempest-DeleteServersAdminTestJSON-570518421-project-member] Enqueued Resource ID: 86656952-d254-40e4-a34d-b4f64574ede0 with Priority: HIGHEST for action security_group_members
[  530.996417] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 86656952-d254-40e4-a34d-b4f64574ede0 with Priority: HIGHEST for action security_group_members
[  530.996417] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.996417] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-86656952-d254-40e4-a34d-b4f64574ede0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.996417] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  530.996417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  530.996417] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('86656952-d254-40e4-a34d-b4f64574ede0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  530.996960] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 17879adb87bb482ab1e28f5eb8e0ecb8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  531.008804] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 17879adb87bb482ab1e28f5eb8e0ecb8
[  531.100761] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.100958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.101078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.101340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.101476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.101720] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.101876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.101988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.102255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.102798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.102957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.103145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.104176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.104176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.104176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.104176] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.104176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.104176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.104432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.104570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.104843] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.104950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.104994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.105297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.105397] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  531.105816] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4eb4cf8276454bfa9c52e2d8ecb10d6a in queue reply_156db6c4256a4c58884afce15e6f5340
[  531.114132] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4eb4cf8276454bfa9c52e2d8ecb10d6a
[  531.187178] env[60890]: INFO networking_nsxv3.common.synchronization [req-1bdaeff2-4879-4c08-bae9-b1e0444b2a04 req-a9b634a5-5fcf-46a0-8820-09a265f72b93 tempest-FloatingIPsAssociationTestJSON-1051505831 tempest-FloatingIPsAssociationTestJSON-1051505831-project-member] Enqueued Resource ID: 014641c6-6987-45ee-8ee5-ed1bf025b019 with Priority: HIGHEST for action security_group_members
[  531.187692] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 014641c6-6987-45ee-8ee5-ed1bf025b019 with Priority: HIGHEST for action security_group_members
[  531.188041] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.188135] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.188204] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.188403] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.188482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.188628] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-014641c6-6987-45ee-8ee5-ed1bf025b019 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.188725] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-014641c6-6987-45ee-8ee5-ed1bf025b019" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.188789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-014641c6-6987-45ee-8ee5-ed1bf025b019" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.188929] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-014641c6-6987-45ee-8ee5-ed1bf025b019" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.189061] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.189151] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.189198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.189356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.189436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.189515] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('014641c6-6987-45ee-8ee5-ed1bf025b019',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  531.189695] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2525bf9befd442769bf86bdae8cfc724 in queue reply_156db6c4256a4c58884afce15e6f5340
[  531.198815] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.198997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.199116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.201396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.201546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.201820] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.201973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.202083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.202303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.202665] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.202856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.203028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.203200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.203337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.203481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.203745] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.206224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.206224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.206224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.206224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.206224] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.206224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.206224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.206224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.206224] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  531.206224] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 96395809f83b460ba68fa2b8c3ff776f in queue reply_156db6c4256a4c58884afce15e6f5340
[  531.207154] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2525bf9befd442769bf86bdae8cfc724
[  531.212850] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 96395809f83b460ba68fa2b8c3ff776f
[  531.244294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-a744ac9f-9a7e-47a8-b2a0-0d86311039f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.244533] env[60890]: Traceback (most recent call last):
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  531.244578] env[60890]:     timer()
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  531.244578] env[60890]:     cb(*args, **kw)
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  531.244578] env[60890]:     result = function(*args, **kwargs)
[  531.244578] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  531.244578] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  531.244578] env[60890]:     return method(*args, **kwargs)
[  531.244578] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  531.244578] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  531.244578] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  531.244578] env[60890]:     result = self.transport._send(
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  531.244578] env[60890]:     return self._driver.send(target, ctxt, message,
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  531.244578] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  531.244578] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  531.244578] env[60890]:     raise result
[  531.245194] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.245194] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.438732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.439266] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.439266] env[60890]: ERROR oslo_messaging.rpc.server 
[  531.499820] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.499959] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.500245] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.500245] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.500557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.500557] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.500651] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.500651] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.501245] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.501245] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.501245] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 86656952-d254-40e4-a34d-b4f64574ede0 already deleted.
[  531.501245] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.608260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.608622] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.608622] env[60890]: ERROR oslo_messaging.rpc.server 
[  531.695788] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.696179] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.696179] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.696414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.696511] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.696700] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.696803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.696862] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.697012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.697086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.697170] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 014641c6-6987-45ee-8ee5-ed1bf025b019 already deleted.
[  531.697245] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-014641c6-6987-45ee-8ee5-ed1bf025b019" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.711894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.712518] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.712518] env[60890]: ERROR oslo_messaging.rpc.server 
[  531.737033] env[60890]: INFO networking_nsxv3.common.synchronization [None req-ba950b9a-eacb-45f0-a5b7-ed043751d43f None None] Enqueued Resource ID: 1a4cb3f0-5434-4715-af33-0e1295d9750e with Priority: HIGHEST for action port
[  531.737312] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 1a4cb3f0-5434-4715-af33-0e1295d9750e with Priority: HIGHEST for action port
[  531.737875] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.738003] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.738070] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.738272] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.738353] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.738502] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-1a4cb3f0-5434-4715-af33-0e1295d9750e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.738606] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-1a4cb3f0-5434-4715-af33-0e1295d9750e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.738692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-1a4cb3f0-5434-4715-af33-0e1295d9750e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.738897] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-1a4cb3f0-5434-4715-af33-0e1295d9750e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.738978] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1a4cb3f0-5434-4715-af33-0e1295d9750e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  531.739160] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8847ac49117e40679a4a0d43e4088e8a in queue reply_156db6c4256a4c58884afce15e6f5340
[  531.745939] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8847ac49117e40679a4a0d43e4088e8a
[  531.749292] env[60890]: INFO networking_nsxv3.common.synchronization [req-d8acbc98-722d-4cca-bf4e-148ee0510705 req-f57c8404-27cb-4f87-b9dd-c51fd8e883de tempest-FloatingIPsAssociationNegativeTestJSON-680885566 tempest-FloatingIPsAssociationNegativeTestJSON-680885566-project-member] Enqueued Resource ID: d701bc90-6679-4fa4-b21b-e638b2b64596 with Priority: HIGHEST for action security_group_members
[  531.749534] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d701bc90-6679-4fa4-b21b-e638b2b64596 with Priority: HIGHEST for action security_group_members
[  531.750245] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.750365] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.750431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.750636] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.750721] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.750898] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-d701bc90-6679-4fa4-b21b-e638b2b64596 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.750999] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-d701bc90-6679-4fa4-b21b-e638b2b64596" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.751215] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-d701bc90-6679-4fa4-b21b-e638b2b64596" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.751215] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-d701bc90-6679-4fa4-b21b-e638b2b64596" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.751339] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.751426] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.751483] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.751630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.751690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.751776] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('d701bc90-6679-4fa4-b21b-e638b2b64596',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  531.751969] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e8fc44fa7fa54550808a3c824fa58354 in queue reply_156db6c4256a4c58884afce15e6f5340
[  531.761191] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e8fc44fa7fa54550808a3c824fa58354
[  531.785403] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.785486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.786329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.786750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.786929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.787241] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.787465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.787584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.787820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.788541] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.788709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.788827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.789043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.789181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.789359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.789593] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.789760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.789879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.790095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.790227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.790471] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7c7ca890-a60b-4bde-8972-38539ca26d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.790627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.790836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.791058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.791340] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7c7ca890-a60b-4bde-8972-38539ca26d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  531.791412] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6f02225727cb4199b1f1d24b9ace3840 in queue reply_156db6c4256a4c58884afce15e6f5340
[  531.798020] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6f02225727cb4199b1f1d24b9ace3840
[  531.806266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7c7ca890-a60b-4bde-8972-38539ca26d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.807740] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  531.807740] env[60890]: ERROR oslo_messaging.rpc.server 
[  531.901735] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.901735] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.901735] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.901735] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  531.901735] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  531.901735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  531.901735] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  531.901735] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f582d7e4f07c45e7a40c4a6f1f4b0a60 in queue reply_156db6c4256a4c58884afce15e6f5340
[  531.907386] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f582d7e4f07c45e7a40c4a6f1f4b0a60
[  532.025487] env[60890]: INFO networking_nsxv3.common.synchronization [req-484b83ed-8792-43ac-9941-0470349490c7 req-90db766d-cde2-4895-ad2c-097ccf494095 tempest-TenantUsagesTestJSON-1613839861 tempest-TenantUsagesTestJSON-1613839861-project-member] Enqueued Resource ID: 27022d8e-fbf0-46b2-bdab-d93cd9cc0c46 with Priority: HIGHEST for action security_group_members
[  532.025487] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 27022d8e-fbf0-46b2-bdab-d93cd9cc0c46 with Priority: HIGHEST for action security_group_members
[  532.027014] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.027014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.027014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.027014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.027014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.027014] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.027014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.027014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.027014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.027014] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.027014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.027477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.027477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.027477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.027477] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('27022d8e-fbf0-46b2-bdab-d93cd9cc0c46',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.027794] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4b3de66f6bce4c8aa4800a3022a7edd3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.041532] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4b3de66f6bce4c8aa4800a3022a7edd3
[  532.141286] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 125 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.141286] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.141286] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.141286] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.190023] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.198097] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Security Group (Members) Is:0.
[  532.198198] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Security Group (Members) Are:set() {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  532.198396] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Inventory metadata have been refreshed.
[  532.198474] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Realizing 20/96 resources of Type:Ports
[  532.198555] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 519bac90-d40c-494c-8eb9-6c53e239f10a with Priority: HIGH for action port
[  532.198639] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 32332759-ca87-4ef6-955a-325ad700200d with Priority: HIGH for action port
[  532.198709] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 03fdd0ff-044d-4309-905b-d51a3bab1b81 with Priority: HIGH for action port
[  532.198767] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 55f117f0-ba76-491f-b5c5-7441d3ff7787 with Priority: HIGH for action port
[  532.198827] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 9435dc4a-ead0-4afd-a099-668739ae29f4 with Priority: HIGH for action port
[  532.198887] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 66d98031-6842-491c-9961-c1b4d0fd5d53 with Priority: HIGH for action port
[  532.198944] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 0f875a82-d52b-4617-86ef-09c6a8aeccbb with Priority: HIGH for action port
[  532.199003] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 2126120d-06ba-46a5-a74c-77f3b18345ab with Priority: HIGH for action port
[  532.199060] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 17b157f7-bf15-4a5c-a271-9af72764322d with Priority: HIGH for action port
[  532.199119] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 474946e7-99a3-4290-959e-f2dc1645b999 with Priority: HIGH for action port
[  532.199175] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 4fb6c40c-bf99-4b55-9f6b-ca451ded5287 with Priority: HIGH for action port
[  532.199232] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 7e152039-3f45-45d1-b2a1-c7acb717f68b with Priority: HIGH for action port
[  532.199289] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 1ca72486-d800-43c5-907b-725bb958b3a3 with Priority: HIGH for action port
[  532.199347] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 2a1bc88a-7641-4017-874c-f15ee8ec3621 with Priority: HIGH for action port
[  532.199408] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: d357f4a7-7fe1-4677-a32d-b2a6c5780592 with Priority: HIGH for action port
[  532.199602] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 99c860bb-14da-4e95-b94e-d77d15224b7a with Priority: HIGH for action port
[  532.199602] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: fb2c7956-257f-4d1d-8dbb-83b5350798c3 with Priority: HIGH for action port
[  532.199602] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: c4585efa-40da-4b2f-81a8-523efec14826 with Priority: HIGH for action port
[  532.199705] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 7e3eabc0-1383-412e-a875-c4e73daf6808 with Priority: HIGH for action port
[  532.199705] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: f599de71-cb96-446c-b853-bfcdc671260e with Priority: HIGH for action port
[  532.199804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.202371] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.202640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.202707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.202920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.203055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.203302] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.203452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.203565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.203774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.204023] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.204197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.204313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.204515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.204673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.204818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.205066] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.205215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.205327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.205526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.205661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.205916] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.206065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.206176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.206375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.206501] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.207839] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1208b7d3c1de4ad9addf30b025b473ab in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.214997] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1208b7d3c1de4ad9addf30b025b473ab
[  532.242061] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-1a4cb3f0-5434-4715-af33-0e1295d9750e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.242706] env[60890]: Traceback (most recent call last):
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  532.242768] env[60890]:     timer()
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  532.242768] env[60890]:     cb(*args, **kw)
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  532.242768] env[60890]:     result = function(*args, **kwargs)
[  532.242768] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  532.242768] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  532.242768] env[60890]:     return method(*args, **kwargs)
[  532.242768] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  532.242768] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  532.242768] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  532.242768] env[60890]:     result = self.transport._send(
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  532.242768] env[60890]:     return self._driver.send(target, ctxt, message,
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  532.242768] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  532.242768] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  532.242768] env[60890]:     raise result
[  532.243570] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  532.243570] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  532.255321] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.255452] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.255518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.256148] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.256259] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.256728] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.256728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.256849] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.256917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.256989] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.257082] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: d701bc90-6679-4fa4-b21b-e638b2b64596 already deleted.
[  532.257162] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-d701bc90-6679-4fa4-b21b-e638b2b64596" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.257975] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.258133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.258246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.258611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.258714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.259582] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.259582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.259582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.259582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.260138] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.260299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.260413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.260616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.260859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.260920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.261116] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.261258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.261367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.261561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.261716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.261991] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.262129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.262237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.262431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.262734] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.262993] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 583f6ebb68324c91b838b4291e3b67b1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.272371] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 583f6ebb68324c91b838b4291e3b67b1
[  532.402283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  532.402813] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  532.402813] env[60890]: ERROR oslo_messaging.rpc.server 
[  532.530034] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.530185] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.530261] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.530484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.530576] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.530811] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.530921] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.530983] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.531134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.531214] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.531299] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 27022d8e-fbf0-46b2-bdab-d93cd9cc0c46 already deleted.
[  532.531378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.710199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  532.710780] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  532.710780] env[60890]: ERROR oslo_messaging.rpc.server 
[  532.765395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  532.765915] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  532.765915] env[60890]: ERROR oslo_messaging.rpc.server 
[  532.828323] env[60890]: INFO networking_nsxv3.common.synchronization [None req-f3853e6a-0b89-45d8-802e-7e2175ac8208 tempest-VolumesAssistedSnapshotsTest-1202454958 tempest-VolumesAssistedSnapshotsTest-1202454958-project-member] Enqueued Resource ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 with Priority: HIGHEST for action security_group_rules
[  532.828842] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 with Priority: HIGHEST for action security_group_rules
[  532.828988] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 519bac90-d40c-494c-8eb9-6c53e239f10a with Priority: HIGH for action port
[  532.829091] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 03fdd0ff-044d-4309-905b-d51a3bab1b81 with Priority: HIGH for action port
[  532.829185] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0f875a82-d52b-4617-86ef-09c6a8aeccbb with Priority: HIGH for action port
[  532.829792] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d357f4a7-7fe1-4677-a32d-b2a6c5780592 with Priority: HIGH for action port
[  532.829947] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f599de71-cb96-446c-b853-bfcdc671260e with Priority: HIGH for action port
[  532.830054] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7e3eabc0-1383-412e-a875-c4e73daf6808 with Priority: HIGH for action port
[  532.830149] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c4585efa-40da-4b2f-81a8-523efec14826 with Priority: HIGH for action port
[  532.830244] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: fb2c7956-257f-4d1d-8dbb-83b5350798c3 with Priority: HIGH for action port
[  532.830332] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2a1bc88a-7641-4017-874c-f15ee8ec3621 with Priority: HIGH for action port
[  532.830421] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 99c860bb-14da-4e95-b94e-d77d15224b7a with Priority: HIGH for action port
[  532.830510] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 1ca72486-d800-43c5-907b-725bb958b3a3 with Priority: HIGH for action port
[  532.830595] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7e152039-3f45-45d1-b2a1-c7acb717f68b with Priority: HIGH for action port
[  532.830680] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 4fb6c40c-bf99-4b55-9f6b-ca451ded5287 with Priority: HIGH for action port
[  532.830762] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 474946e7-99a3-4290-959e-f2dc1645b999 with Priority: HIGH for action port
[  532.830842] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 17b157f7-bf15-4a5c-a271-9af72764322d with Priority: HIGH for action port
[  532.830923] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 66d98031-6842-491c-9961-c1b4d0fd5d53 with Priority: HIGH for action port
[  532.831012] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2126120d-06ba-46a5-a74c-77f3b18345ab with Priority: HIGH for action port
[  532.831083] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9435dc4a-ead0-4afd-a099-668739ae29f4 with Priority: HIGH for action port
[  532.831169] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 32332759-ca87-4ef6-955a-325ad700200d with Priority: HIGH for action port
[  532.831256] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 55f117f0-ba76-491f-b5c5-7441d3ff7787 with Priority: HIGH for action port
[  532.831715] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.831715] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.831715] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.831877] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.831954] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.832113] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.832202] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.832259] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.833212] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.833608] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('6cb50a73-5b5c-401a-b812-ccab4721d4a6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.833608] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7a7ee48fe0034f69aac3ccf481e2fd16 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.834349] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.834519] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.834658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.834874] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.835025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.835218] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-519bac90-d40c-494c-8eb9-6c53e239f10a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.835361] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.835461] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.835696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.835827] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('519bac90-d40c-494c-8eb9-6c53e239f10a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.836041] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 76f2b8b7781f401f812fad3014e6a9b9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.836727] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.836883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.836984] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.837213] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.837372] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.837569] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-03fdd0ff-044d-4309-905b-d51a3bab1b81 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.837728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.838346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.838346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.838346] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03fdd0ff-044d-4309-905b-d51a3bab1b81',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.838346] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 917d5e1c3be3412c90804e85c19628f4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.839406] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.839839] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.839959] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.840168] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.840294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.840478] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-0f875a82-d52b-4617-86ef-09c6a8aeccbb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.840617] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.840726] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.841017] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.841316] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f875a82-d52b-4617-86ef-09c6a8aeccbb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.841542] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a2d4455026ca42ccbcbe0ef430afdb08 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.843010] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.843191] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.843299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.843766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.843863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.844675] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-d357f4a7-7fe1-4677-a32d-b2a6c5780592 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.844879] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.844994] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.845232] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.845352] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d357f4a7-7fe1-4677-a32d-b2a6c5780592',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.845545] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d165a85d26ce495bbe1253aa5ba241e4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.846806] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.846894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.847021] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.847480] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.847619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.847808] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-f599de71-cb96-446c-b853-bfcdc671260e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.849256] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.849256] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.849256] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.849256] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f599de71-cb96-446c-b853-bfcdc671260e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.849929] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a57ef04e464e45b7a37c0ee740409b94 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.851006] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.851006] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.851100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.851278] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.851405] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.851594] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-7e3eabc0-1383-412e-a875-c4e73daf6808 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.851769] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.851881] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.852140] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.852265] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e3eabc0-1383-412e-a875-c4e73daf6808',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.852482] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg cb255e688f1b49f6b9f091fecfca90d2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.853433] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.853591] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.853702] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.853986] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.854602] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.854602] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-c4585efa-40da-4b2f-81a8-523efec14826 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.854602] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.854602] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.854602] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.854949] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4585efa-40da-4b2f-81a8-523efec14826',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.855056] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8ad5e99256ed4537bafa3e51688c259d in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.855880] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.856035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.856147] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.856349] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.856478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.856677] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-fb2c7956-257f-4d1d-8dbb-83b5350798c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.856822] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.856923] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.857138] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.857258] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fb2c7956-257f-4d1d-8dbb-83b5350798c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.857441] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d56e395312f64e9f8f24ddf4bc0b9d93 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.858258] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.858417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.858519] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.858708] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.858841] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.859023] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2a1bc88a-7641-4017-874c-f15ee8ec3621 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.859157] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.859251] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.859456] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.859572] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2a1bc88a-7641-4017-874c-f15ee8ec3621',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.859748] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7cb8fa02285a4b70b3309db311519666 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.860810] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.860810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.860931] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.861101] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.861222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.861434] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-99c860bb-14da-4e95-b94e-d77d15224b7a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.861570] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.861665] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.862036] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.862036] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('99c860bb-14da-4e95-b94e-d77d15224b7a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.862137] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0f15caf0ef4346bead80cf790c5ff191 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.862360] env[60890]: WARNING oslo_messaging._drivers.amqpdriver [-] Number of call queues is 11, greater than warning threshold: 10. There could be a leak. Increasing threshold to: 20
[  532.863109] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.863272] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.863372] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.863562] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.863695] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.863904] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-1ca72486-d800-43c5-907b-725bb958b3a3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.864165] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.864287] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.864497] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.864614] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1ca72486-d800-43c5-907b-725bb958b3a3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.864817] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d104aaba213047309c689abb0f0cadb9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.865638] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.865795] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.865986] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.866178] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.866321] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.866504] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-7e152039-3f45-45d1-b2a1-c7acb717f68b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.866644] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.866759] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.867020] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.867141] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e152039-3f45-45d1-b2a1-c7acb717f68b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.867324] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5bcadb6af5184bda9f90acb21a035917 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.868513] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.868680] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.868790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.868983] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.869314] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.869619] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.869619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.869750] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.870407] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.870407] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4fb6c40c-bf99-4b55-9f6b-ca451ded5287',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.870838] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5de0a828afb8481fae146c8c815b4f43 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.872832] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.873236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.873404] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.873696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.873821] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.874294] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-474946e7-99a3-4290-959e-f2dc1645b999 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.874467] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.874576] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.874830] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.874959] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('474946e7-99a3-4290-959e-f2dc1645b999',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.875144] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 932bc704dffd496592d4c668fe95ed7a in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.875961] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.876140] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.876244] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.876442] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.876571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.876761] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-17b157f7-bf15-4a5c-a271-9af72764322d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.876918] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.877019] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.877359] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.877359] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17b157f7-bf15-4a5c-a271-9af72764322d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.878119] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b90695e0ff3f4b478f23dbf717491c21 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.878797] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.878797] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.878797] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.878797] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.878797] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.879544] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-66d98031-6842-491c-9961-c1b4d0fd5d53 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.879544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.879544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.879544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.879544] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('66d98031-6842-491c-9961-c1b4d0fd5d53',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.879841] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 92290975b49043eeae36bcd74fa2249a in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.881310] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.881310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.881310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.881310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.881310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.881310] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2126120d-06ba-46a5-a74c-77f3b18345ab taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.881310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.881310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.881670] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.881670] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2126120d-06ba-46a5-a74c-77f3b18345ab',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.881850] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 80c42725deff4decbaa29748b72ef0ea in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.883039] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.883239] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.883346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.883571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.883714] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.883899] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-9435dc4a-ead0-4afd-a099-668739ae29f4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.884119] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.884292] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.884891] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.884891] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9435dc4a-ead0-4afd-a099-668739ae29f4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.885554] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5f8f2f9c23bc42c1a55759ec9b5584c3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.886475] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.886643] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.886755] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.887067] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.887184] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.887375] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-32332759-ca87-4ef6-955a-325ad700200d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.887518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.887618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.887832] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.887953] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32332759-ca87-4ef6-955a-325ad700200d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.888165] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7138e0af70984dbbb2a33229fc1712ed in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.889448] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.890481] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.890606] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.890807] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.891067] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.891116] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-55f117f0-ba76-491f-b5c5-7441d3ff7787 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.891250] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.891353] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.891562] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.891680] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('55f117f0-ba76-491f-b5c5-7441d3ff7787',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.891873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 658559798c164a11a728d3561f91e42b in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.892068] env[60890]: WARNING oslo_messaging._drivers.amqpdriver [-] Number of call queues is 21, greater than warning threshold: 20. There could be a leak. Increasing threshold to: 40
[  532.893123] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 76f2b8b7781f401f812fad3014e6a9b9
[  532.894470] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7a7ee48fe0034f69aac3ccf481e2fd16
[  532.898757] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.899291] env[60890]: Traceback (most recent call last):
[  532.899361] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/poll.py", line 111, in wait
[  532.899361] env[60890]:     listener.cb(fileno)
[  532.899361] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  532.899361] env[60890]:     result = function(*args, **kwargs)
[  532.899361] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  532.899361] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  532.899361] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  532.899361] env[60890]:     return method(*args, **kwargs)
[  532.899361] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  532.899361] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  532.899361] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  532.899361] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  532.899361] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  532.899361] env[60890]:     result = self.transport._send(
[  532.899361] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  532.899361] env[60890]:     return self._driver.send(target, ctxt, message,
[  532.899361] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  532.899361] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  532.899361] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  532.899361] env[60890]:     raise result
[  532.899361] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  532.899361] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  532.899361] env[60890]: Removing descriptor: 17
[  532.921399] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.921596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.921718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.921988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.922127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.922374] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.922555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.922646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.922888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.923100] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.923257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.923384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.923587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.923774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.923877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.924118] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.924268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.924381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.924589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.924746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.924974] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.925124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.925237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.925489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.925634] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  532.925884] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 99556ff441b54cffa309265f8cacc196 in queue reply_156db6c4256a4c58884afce15e6f5340
[  532.928707] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  532.928707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  532.928707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  532.928707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  532.929133] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.929133] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 already deleted.
[  532.929133] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  532.933483] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 99556ff441b54cffa309265f8cacc196
[  532.946431] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d104aaba213047309c689abb0f0cadb9
[  532.946944] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 932bc704dffd496592d4c668fe95ed7a
[  532.959375] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d165a85d26ce495bbe1253aa5ba241e4
[  533.041959] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7138e0af70984dbbb2a33229fc1712ed
[  533.042842] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 658559798c164a11a728d3561f91e42b
[  533.050885] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5de0a828afb8481fae146c8c815b4f43
[  533.051375] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b90695e0ff3f4b478f23dbf717491c21
[  533.051790] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cb255e688f1b49f6b9f091fecfca90d2
[  533.052232] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5bcadb6af5184bda9f90acb21a035917
[  533.055265] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8ad5e99256ed4537bafa3e51688c259d
[  533.055265] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 92290975b49043eeae36bcd74fa2249a
[  533.055265] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a57ef04e464e45b7a37c0ee740409b94
[  533.055265] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 917d5e1c3be3412c90804e85c19628f4
[  533.055265] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d56e395312f64e9f8f24ddf4bc0b9d93
[  533.055265] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7cb8fa02285a4b70b3309db311519666
[  533.055265] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 80c42725deff4decbaa29748b72ef0ea
[  533.056209] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0f15caf0ef4346bead80cf790c5ff191
[  533.056209] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5f8f2f9c23bc42c1a55759ec9b5584c3
[  533.057537] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2d4455026ca42ccbcbe0ef430afdb08
[  533.097425] env[60890]: INFO networking_nsxv3.common.synchronization [req-fa05baee-4514-4bb3-ab39-ebb0b2fdc414 req-6374585b-48c4-4f37-8755-49c53f73c703 tempest-MigrationsAdminTest-320299187 tempest-MigrationsAdminTest-320299187-project-member] Enqueued Resource ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b with Priority: HIGHEST for action security_group_members
[  533.098028] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b with Priority: HIGHEST for action security_group_members
[  533.098110] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.098787] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.098787] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.098787] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.098787] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.098787] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-18ba6c5b-4074-4f30-b957-61d36273ed7b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.099092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.099092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.099092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.099186] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.099282] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.099327] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.099672] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.099672] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.099672] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('18ba6c5b-4074-4f30-b957-61d36273ed7b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  533.099808] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5b7ecdda19a1443a9a33041b3fdc2b8f in queue reply_156db6c4256a4c58884afce15e6f5340
[  533.108389] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5b7ecdda19a1443a9a33041b3fdc2b8f
[  533.194585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.194585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.194585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.194585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.194585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.194585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.194585] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  533.194585] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 55cf658b27e04d69b5e2c508e59d7b81 in queue reply_156db6c4256a4c58884afce15e6f5340
[  533.199156] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 55cf658b27e04d69b5e2c508e59d7b81
[  533.413495] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.413495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.413495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.413495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.413495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.413817] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.413817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.414374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.414452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.414680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.414851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.414968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.415175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.415313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.416084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.416084] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.416084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.416084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.416285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.416336] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.416573] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.416758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.416878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.417113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.417259] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  533.417493] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4edc4c7cc6f54f3ebfdd6487965ad136 in queue reply_156db6c4256a4c58884afce15e6f5340
[  533.424401] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4edc4c7cc6f54f3ebfdd6487965ad136
[  533.430828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.431286] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.431286] env[60890]: ERROR oslo_messaging.rpc.server 
[  533.442763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.443155] env[60890]: Traceback (most recent call last):
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.443155] env[60890]:     timer()
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.443155] env[60890]:     cb(*args, **kw)
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.443155] env[60890]:     result = function(*args, **kwargs)
[  533.443155] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.443155] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.443155] env[60890]:     return method(*args, **kwargs)
[  533.443155] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.443155] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.443155] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.443155] env[60890]:     result = self.transport._send(
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.443155] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.443155] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.443155] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.443155] env[60890]:     raise result
[  533.443155] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.443155] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.443877] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.443877] env[60890]: Traceback (most recent call last):
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.443877] env[60890]:     timer()
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.443877] env[60890]:     cb(*args, **kw)
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.443877] env[60890]:     result = function(*args, **kwargs)
[  533.443877] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.443877] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.443877] env[60890]:     return method(*args, **kwargs)
[  533.443877] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.443877] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.443877] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.443877] env[60890]:     result = self.transport._send(
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.443877] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.443877] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.443877] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.443877] env[60890]:     raise result
[  533.443877] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.443877] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.443877] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.443877] env[60890]: Traceback (most recent call last):
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.444995] env[60890]:     timer()
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.444995] env[60890]:     cb(*args, **kw)
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.444995] env[60890]:     result = function(*args, **kwargs)
[  533.444995] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.444995] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.444995] env[60890]:     return method(*args, **kwargs)
[  533.444995] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.444995] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.444995] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.444995] env[60890]:     result = self.transport._send(
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.444995] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.444995] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.444995] env[60890]:     raise result
[  533.444995] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.444995] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.444995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.444995] env[60890]: Traceback (most recent call last):
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.444995] env[60890]:     timer()
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.444995] env[60890]:     cb(*args, **kw)
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.444995] env[60890]:     result = function(*args, **kwargs)
[  533.444995] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.444995] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.444995] env[60890]:     return method(*args, **kwargs)
[  533.444995] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.444995] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.444995] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.444995] env[60890]:     result = self.transport._send(
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.444995] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.444995] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.444995] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.444995] env[60890]:     raise result
[  533.444995] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.444995] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.448471] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.448471] env[60890]: Traceback (most recent call last):
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.448471] env[60890]:     timer()
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.448471] env[60890]:     cb(*args, **kw)
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.448471] env[60890]:     result = function(*args, **kwargs)
[  533.448471] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.448471] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.448471] env[60890]:     return method(*args, **kwargs)
[  533.448471] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.448471] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.448471] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.448471] env[60890]:     result = self.transport._send(
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.448471] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.448471] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.448471] env[60890]:     raise result
[  533.448471] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.448471] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.448471] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.448471] env[60890]: Traceback (most recent call last):
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.448471] env[60890]:     timer()
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.448471] env[60890]:     cb(*args, **kw)
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.448471] env[60890]:     result = function(*args, **kwargs)
[  533.448471] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.448471] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.448471] env[60890]:     return method(*args, **kwargs)
[  533.448471] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.448471] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.448471] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.448471] env[60890]:     result = self.transport._send(
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.448471] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.448471] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.448471] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.448471] env[60890]:     raise result
[  533.448471] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.448471] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.450166] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.450166] env[60890]: Traceback (most recent call last):
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.450166] env[60890]:     timer()
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.450166] env[60890]:     cb(*args, **kw)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.450166] env[60890]:     result = function(*args, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.450166] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.450166] env[60890]:     return method(*args, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.450166] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.450166] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.450166] env[60890]:     result = self.transport._send(
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.450166] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.450166] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.450166] env[60890]:     raise result
[  533.450166] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.450166] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.450166] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.450166] env[60890]: Traceback (most recent call last):
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.450166] env[60890]:     timer()
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.450166] env[60890]:     cb(*args, **kw)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.450166] env[60890]:     result = function(*args, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.450166] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.450166] env[60890]:     return method(*args, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.450166] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.450166] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.450166] env[60890]:     result = self.transport._send(
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.450166] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.450166] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.450166] env[60890]:     raise result
[  533.450166] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.450166] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.450166] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.450166] env[60890]: Traceback (most recent call last):
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.450166] env[60890]:     timer()
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.450166] env[60890]:     cb(*args, **kw)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.450166] env[60890]:     result = function(*args, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.450166] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.450166] env[60890]:     return method(*args, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.450166] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.450166] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.450166] env[60890]:     result = self.transport._send(
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.450166] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.450166] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.450166] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.450166] env[60890]:     raise result
[  533.450166] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.450166] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.453363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.453363] env[60890]: Traceback (most recent call last):
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.453363] env[60890]:     timer()
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.453363] env[60890]:     cb(*args, **kw)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.453363] env[60890]:     result = function(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.453363] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.453363] env[60890]:     return method(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.453363] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.453363] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.453363] env[60890]:     result = self.transport._send(
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.453363] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.453363] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.453363] env[60890]:     raise result
[  533.453363] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.453363] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.453363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.453363] env[60890]: Traceback (most recent call last):
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.453363] env[60890]:     timer()
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.453363] env[60890]:     cb(*args, **kw)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.453363] env[60890]:     result = function(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.453363] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.453363] env[60890]:     return method(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.453363] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.453363] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.453363] env[60890]:     result = self.transport._send(
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.453363] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.453363] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.453363] env[60890]:     raise result
[  533.453363] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.453363] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.453363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.453363] env[60890]: Traceback (most recent call last):
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.453363] env[60890]:     timer()
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.453363] env[60890]:     cb(*args, **kw)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.453363] env[60890]:     result = function(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.453363] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.453363] env[60890]:     return method(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.453363] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.453363] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.453363] env[60890]:     result = self.transport._send(
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.453363] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.453363] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.453363] env[60890]:     raise result
[  533.453363] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.453363] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.453363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.453363] env[60890]: Traceback (most recent call last):
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.453363] env[60890]:     timer()
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.453363] env[60890]:     cb(*args, **kw)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.453363] env[60890]:     result = function(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.453363] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.453363] env[60890]:     return method(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.453363] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.453363] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.453363] env[60890]:     result = self.transport._send(
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.453363] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.453363] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.453363] env[60890]:     raise result
[  533.453363] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.453363] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.453363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.453363] env[60890]: Traceback (most recent call last):
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.453363] env[60890]:     timer()
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.453363] env[60890]:     cb(*args, **kw)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.453363] env[60890]:     result = function(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.453363] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.453363] env[60890]:     return method(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.453363] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.453363] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.453363] env[60890]:     result = self.transport._send(
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.453363] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.453363] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.453363] env[60890]:     raise result
[  533.453363] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.453363] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.453363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.453363] env[60890]: Traceback (most recent call last):
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.453363] env[60890]:     timer()
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.453363] env[60890]:     cb(*args, **kw)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.453363] env[60890]:     result = function(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.453363] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.453363] env[60890]:     return method(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.453363] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.453363] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.453363] env[60890]:     result = self.transport._send(
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.453363] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.453363] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.453363] env[60890]:     raise result
[  533.453363] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.453363] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.453363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.453363] env[60890]: Traceback (most recent call last):
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.453363] env[60890]:     timer()
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.453363] env[60890]:     cb(*args, **kw)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.453363] env[60890]:     result = function(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.453363] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.453363] env[60890]:     return method(*args, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.453363] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.453363] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.453363] env[60890]:     result = self.transport._send(
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.453363] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.453363] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.453363] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.453363] env[60890]:     raise result
[  533.453363] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.453363] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.453363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.459388] env[60890]: Traceback (most recent call last):
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.459388] env[60890]:     timer()
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.459388] env[60890]:     cb(*args, **kw)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.459388] env[60890]:     result = function(*args, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.459388] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.459388] env[60890]:     return method(*args, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.459388] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.459388] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.459388] env[60890]:     result = self.transport._send(
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.459388] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.459388] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.459388] env[60890]:     raise result
[  533.459388] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.459388] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.459388] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.459388] env[60890]: Traceback (most recent call last):
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.459388] env[60890]:     timer()
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.459388] env[60890]:     cb(*args, **kw)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.459388] env[60890]:     result = function(*args, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.459388] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.459388] env[60890]:     return method(*args, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.459388] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.459388] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.459388] env[60890]:     result = self.transport._send(
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.459388] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.459388] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.459388] env[60890]:     raise result
[  533.459388] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.459388] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.459388] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.459388] env[60890]: Traceback (most recent call last):
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  533.459388] env[60890]:     timer()
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  533.459388] env[60890]:     cb(*args, **kw)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  533.459388] env[60890]:     result = function(*args, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  533.459388] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.459388] env[60890]:     return method(*args, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.459388] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.459388] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.459388] env[60890]:     result = self.transport._send(
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.459388] env[60890]:     return self._driver.send(target, ctxt, message,
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.459388] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.459388] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.459388] env[60890]:     raise result
[  533.459388] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.459388] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.534962] env[60890]: INFO networking_nsxv3.common.synchronization [req-ed00e15f-3dc4-47d2-bf0a-eee7d7714ed5 req-bf522a62-6a4c-4f1c-9480-17d5028256a1 tempest-ImagesNegativeTestJSON-494836937 tempest-ImagesNegativeTestJSON-494836937-project-member] Enqueued Resource ID: 7663847a-5074-4a6e-81cc-c7458ccf0f4c with Priority: HIGHEST for action security_group_members
[  533.534962] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7663847a-5074-4a6e-81cc-c7458ccf0f4c with Priority: HIGHEST for action security_group_members
[  533.534962] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.534962] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.534962] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.534962] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.534962] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.534962] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7663847a-5074-4a6e-81cc-c7458ccf0f4c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.534962] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7663847a-5074-4a6e-81cc-c7458ccf0f4c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.535659] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7663847a-5074-4a6e-81cc-c7458ccf0f4c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.535659] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7663847a-5074-4a6e-81cc-c7458ccf0f4c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.535659] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.535659] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.535659] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.536047] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.536047] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.536047] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7663847a-5074-4a6e-81cc-c7458ccf0f4c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  533.536595] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg cf5351d67285463d9a5a6111c595a907 in queue reply_156db6c4256a4c58884afce15e6f5340
[  533.549166] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cf5351d67285463d9a5a6111c595a907
[  533.602353] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.602499] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.602569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.602799] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.602887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.603093] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.603179] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.603228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.603372] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.603444] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.603525] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b already deleted.
[  533.603596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.608055] env[60890]: INFO networking_nsxv3.common.synchronization [None req-2ecb44d6-47e0-43a8-9403-22a98b65231f tempest-AttachInterfacesUnderV243Test-1652000262 tempest-AttachInterfacesUnderV243Test-1652000262-project-member] Enqueued Resource ID: 5856cafc-e8df-48e6-a28b-0e951067688f with Priority: HIGHEST for action security_group_rules
[  533.608305] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5856cafc-e8df-48e6-a28b-0e951067688f with Priority: HIGHEST for action security_group_rules
[  533.608841] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.608958] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.609026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.609194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.609276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.609422] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-5856cafc-e8df-48e6-a28b-0e951067688f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.609514] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.609575] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.609786] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.609871] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('5856cafc-e8df-48e6-a28b-0e951067688f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  533.610050] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0f66e86737f349d9898577ffad47e876 in queue reply_156db6c4256a4c58884afce15e6f5340
[  533.621097] env[60890]: INFO networking_nsxv3.common.synchronization [None req-ed187a3f-4203-48c8-ae9c-d213fb8e486a tempest-VolumesAssistedSnapshotsTest-1202454958 tempest-VolumesAssistedSnapshotsTest-1202454958-project-member] Enqueued Resource ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 with Priority: HIGHEST for action security_group_rules
[  533.621658] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0f66e86737f349d9898577ffad47e876
[  533.622101] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 with Priority: HIGHEST for action security_group_rules
[  533.622413] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.622526] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.622589] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.622777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.622898] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.623013] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.623103] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.623159] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.623306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.623377] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('6cb50a73-5b5c-401a-b812-ccab4721d4a6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  533.623549] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e69de1a9a6bf4781b4fe272c5b4e7aed in queue reply_156db6c4256a4c58884afce15e6f5340
[  533.634381] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e69de1a9a6bf4781b4fe272c5b4e7aed
[  533.679531] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.679531] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.679531] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.679531] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.679531] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.679531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.679531] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  533.679531] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2e42ce9b59a946419907ee46c69e88ab in queue reply_156db6c4256a4c58884afce15e6f5340
[  533.693621] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2e42ce9b59a946419907ee46c69e88ab
[  533.694200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.694680] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.694680] env[60890]: ERROR oslo_messaging.rpc.server 
[  533.879782] env[60890]: INFO networking_nsxv3.common.synchronization [None req-9adb56bd-ebf5-44a7-8cb8-f8f9fd852b8f tempest-AttachInterfacesUnderV243Test-1652000262 tempest-AttachInterfacesUnderV243Test-1652000262-project-member] Enqueued Resource ID: 5856cafc-e8df-48e6-a28b-0e951067688f with Priority: HIGHEST for action security_group_rules
[  533.880360] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5856cafc-e8df-48e6-a28b-0e951067688f with Priority: HIGHEST for action security_group_rules
[  533.880686] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.880809] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.881596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  533.881596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  533.881596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.881596] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-5856cafc-e8df-48e6-a28b-0e951067688f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  533.881596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  533.919109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.919650] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  533.919650] env[60890]: ERROR oslo_messaging.rpc.server 
[  534.039054] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.039214] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.039281] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.039499] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.039590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.039778] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.039874] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.039931] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.040101] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.040176] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.040258] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7663847a-5074-4a6e-81cc-c7458ccf0f4c already deleted.
[  534.040335] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7663847a-5074-4a6e-81cc-c7458ccf0f4c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.119823] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.120040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.120173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.120446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.120587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.121048] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.121217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.121341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.121571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.121832] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.121971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.122092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.122299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.122445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.122655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.123307] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.123307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.123307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.123307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.123500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.123676] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.123802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.124062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.124212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.124350] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  534.124586] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f37cff4105f54c3a96f669748af033f0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  534.133759] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.133759] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.133759] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.133759] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.133759] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.133759] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 5856cafc-e8df-48e6-a28b-0e951067688f already deleted.
[  534.133759] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.134730] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f37cff4105f54c3a96f669748af033f0
[  534.135130] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.135344] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.135384] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('5856cafc-e8df-48e6-a28b-0e951067688f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  534.135599] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8523df17ecf4473dbb36bc2147139ce8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  534.142391] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.142391] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.142391] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.142391] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.142391] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.142391] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 already deleted.
[  534.142391] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.150021] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8523df17ecf4473dbb36bc2147139ce8
[  534.154335] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.154529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.154753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.155010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.155161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.155421] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.155570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.155726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.156056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.156315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.156471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.156586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.156808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.156946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.157113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.157494] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.157554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.157657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.157864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.157995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.158232] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.158381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.158496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.158744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.158879] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  534.159107] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 96ae0a980bbd44a089966846706ccb99 in queue reply_156db6c4256a4c58884afce15e6f5340
[  534.168638] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 96ae0a980bbd44a089966846706ccb99
[  534.190253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  534.190725] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  534.190725] env[60890]: ERROR oslo_messaging.rpc.server 
[  534.307682] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.307873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.307995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.308427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.308475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.308753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.309739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.309739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.309739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.309739] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.309739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.309739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.309955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.310013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.310155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.310390] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.310534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.310646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.310843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.310969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.311197] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.311341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.311501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.311656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.311783] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  534.312033] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5553f3905ea44fc691dcd06fe8cb3614 in queue reply_156db6c4256a4c58884afce15e6f5340
[  534.317794] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5553f3905ea44fc691dcd06fe8cb3614
[  534.440531] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f61fc508795c4bf3af198deb82d9d0ea in queue reply_156db6c4256a4c58884afce15e6f5340
[  534.454460] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f61fc508795c4bf3af198deb82d9d0ea
[  534.584669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.584925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.585055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.585326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.585473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.585738] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.585899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.586080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.586309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.586562] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.586718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.586838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.587053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.587186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.587342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.587614] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.587762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.587876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.588095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.588234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.588471] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.588627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.588779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.588993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.589229] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  534.589353] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5cd33eba14a14ffb8cec074be23633d0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  534.598345] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5cd33eba14a14ffb8cec074be23633d0
[  534.627921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  534.628434] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  534.628434] env[60890]: ERROR oslo_messaging.rpc.server 
[  534.654809] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  534.654967] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  534.655044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  534.655252] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  534.655339] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.655430] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 5856cafc-e8df-48e6-a28b-0e951067688f already deleted.
[  534.655510] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.661522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  534.661990] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  534.661990] env[60890]: ERROR oslo_messaging.rpc.server 
[  534.814251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  534.814759] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  534.814759] env[60890]: ERROR oslo_messaging.rpc.server 
[  535.092659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.093182] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.093182] env[60890]: ERROR oslo_messaging.rpc.server 
[  535.155192] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.155390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.155517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.155846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.155988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.156286] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.156446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.156563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.156786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.157090] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.157202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.157288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.158216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.158216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.158216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.158216] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.158216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.158517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.158517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.158580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.158874] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.158976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.159088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.159292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.159421] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  535.159653] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4c69faf86e71478b9383e42acf6975a7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  535.167623] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4c69faf86e71478b9383e42acf6975a7
[  535.304601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.308450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.308638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.308933] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.309083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.309373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.309535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.309650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.309946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.310117] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.310274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.310390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.310598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.310744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.310891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.311154] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.311307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.311419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.311622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.311747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.311984] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.312252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.312252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.312461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.312591] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  535.312898] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 34a0ba192d644b87bb3f273f64d9511d in queue reply_156db6c4256a4c58884afce15e6f5340
[  535.320195] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 34a0ba192d644b87bb3f273f64d9511d
[  535.325601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.325952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.326109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.326417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.326576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.326881] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.327010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.327133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.327353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.327592] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.327741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.327858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.328075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.328215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.328360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.328605] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.328765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.328878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.329091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.329221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.329454] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.329599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.329711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.330152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.330152] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  535.330267] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4ba6b98c3584478799d7eaf62ea4f4d7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  535.349493] env[60890]: INFO networking_nsxv3.common.synchronization [None req-427b5226-2ed6-4d07-b6b7-8e43f80eae02 None None] Enqueued Resource ID: 3e855232-4ef2-4dec-a1d2-2a731ba71939 with Priority: HIGHEST for action port
[  535.350029] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 3e855232-4ef2-4dec-a1d2-2a731ba71939 with Priority: HIGHEST for action port
[  535.350350] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.350479] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.350544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.351773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.351773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.351773] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-3e855232-4ef2-4dec-a1d2-2a731ba71939 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.351773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-3e855232-4ef2-4dec-a1d2-2a731ba71939" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.351773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-3e855232-4ef2-4dec-a1d2-2a731ba71939" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.351773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-3e855232-4ef2-4dec-a1d2-2a731ba71939" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.351773] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3e855232-4ef2-4dec-a1d2-2a731ba71939',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  535.351773] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9a740f06364448078e60205c7787fe4c in queue reply_156db6c4256a4c58884afce15e6f5340
[  535.352502] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4ba6b98c3584478799d7eaf62ea4f4d7
[  535.357175] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.357345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.357518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.357870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.358006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.358664] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.358829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.358948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.359417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.360993] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.360993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.360993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.360993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.360993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.360993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.361364] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.361364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.361364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.361784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.361784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.361908] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.362072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.362172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.362420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.362502] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  535.362967] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4be74ecf953a49bb9204bf4a5d826ccd in queue reply_156db6c4256a4c58884afce15e6f5340
[  535.363938] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9a740f06364448078e60205c7787fe4c
[  535.547825] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4be74ecf953a49bb9204bf4a5d826ccd
[  535.662916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.663498] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.663498] env[60890]: ERROR oslo_messaging.rpc.server 
[  535.789325] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.789536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.789667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.789948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.790100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.790368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.790529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.790648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.790954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.791105] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.791280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.791403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.791614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.791760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.791951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.792243] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.792405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.792534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.792851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.792994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.794105] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.794375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.794464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.794776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.794956] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  535.795209] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3a65ca6293c6423b98d28711c4a5d5a5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  535.801859] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3a65ca6293c6423b98d28711c4a5d5a5
[  535.814808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.815340] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.815340] env[60890]: ERROR oslo_messaging.rpc.server 
[  535.832530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.833154] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.833154] env[60890]: ERROR oslo_messaging.rpc.server 
[  535.854879] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-3e855232-4ef2-4dec-a1d2-2a731ba71939" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.855126] env[60890]: Traceback (most recent call last):
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  535.855171] env[60890]:     timer()
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  535.855171] env[60890]:     cb(*args, **kw)
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  535.855171] env[60890]:     result = function(*args, **kwargs)
[  535.855171] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  535.855171] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  535.855171] env[60890]:     return method(*args, **kwargs)
[  535.855171] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  535.855171] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  535.855171] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  535.855171] env[60890]:     result = self.transport._send(
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  535.855171] env[60890]:     return self._driver.send(target, ctxt, message,
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  535.855171] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  535.855171] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  535.855171] env[60890]:     raise result
[  535.855866] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.855866] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.866294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.866799] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  535.866799] env[60890]: ERROR oslo_messaging.rpc.server 
[  535.992607] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.999246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.999246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.999246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  535.999246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  535.999246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  535.999246] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  535.999246] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 66752904b2ab4ec18ee43c77668867c2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  536.005171] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 66752904b2ab4ec18ee43c77668867c2
[  536.297321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  536.297829] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  536.297829] env[60890]: ERROR oslo_messaging.rpc.server 
[  536.500538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  536.501106] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  536.501106] env[60890]: ERROR oslo_messaging.rpc.server 
[  536.602282] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.602493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.602702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.602990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.603144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.603410] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.603574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.603699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.603910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.604183] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.604342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.604460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.604726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.604886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.605040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.605294] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.605709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.605709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.605816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.605916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.606154] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.606303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.606442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.606617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.606834] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  536.606975] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 14e5773b6e7749dc874fd523a20705b0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  536.611972] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 14e5773b6e7749dc874fd523a20705b0
[  536.751032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.751231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.751357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.751661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.751761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.752023] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.752181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.752302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.752531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.752937] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.752937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.753166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.753869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.753959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.754119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.754479] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.754479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.754627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.754825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.754983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.755216] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.755370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.755488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.755819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.755819] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  536.756072] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg eafdc40de5c04b3285bacd5eaaa593cd in queue reply_156db6c4256a4c58884afce15e6f5340
[  536.758808] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.758979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.759104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.759325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.759464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.759706] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.759893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.759976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.760213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.760696] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.760883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.761008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.761222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.761366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.761702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.761989] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.762203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.762379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.762646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.762755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.762999] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-60131676-f8f4-40d9-8291-263917f353d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.764124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.764124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.764124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.764124] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('60131676-f8f4-40d9-8291-263917f353d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  536.764124] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 28fc9069f47949588434c608ce145c8b in queue reply_156db6c4256a4c58884afce15e6f5340
[  536.767590] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eafdc40de5c04b3285bacd5eaaa593cd
[  536.771027] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 28fc9069f47949588434c608ce145c8b
[  536.778794] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.779012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.779378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.779722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.780085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.780248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.780430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.780529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.780755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.781001] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.781204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.781362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.781583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.781731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.781886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.782137] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.782287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.782410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.782643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.782787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  536.783033] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  536.783190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  536.783308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  536.783616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  536.783771] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  536.784023] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2994f0c5dea6472fbfa2c06b7d00e481 in queue reply_156db6c4256a4c58884afce15e6f5340
[  536.789798] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2994f0c5dea6472fbfa2c06b7d00e481
[  537.091878] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.092106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.092240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.092767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.092767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.093051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.093241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.093366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.093590] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.093836] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.094006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.094125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.094346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.094492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.094677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.094914] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.095068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.095185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.095393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.095526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.095772] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.095923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.096058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.096271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.096407] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  537.096641] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c343c61f74f04bd7874c3ca130967739 in queue reply_156db6c4256a4c58884afce15e6f5340
[  537.101877] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c343c61f74f04bd7874c3ca130967739
[  537.110070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.110504] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.110504] env[60890]: ERROR oslo_messaging.rpc.server 
[  537.266434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.266966] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.266966] env[60890]: ERROR oslo_messaging.rpc.server 
[  537.268311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-60131676-f8f4-40d9-8291-263917f353d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.268607] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.268607] env[60890]: ERROR oslo_messaging.rpc.server 
[  537.289228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.289228] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.289228] env[60890]: ERROR oslo_messaging.rpc.server 
[  537.599288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.599967] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  537.599967] env[60890]: ERROR oslo_messaging.rpc.server 
[  537.604127] env[60890]: INFO networking_nsxv3.common.synchronization [None req-cf6424c1-cc81-45fb-8487-5e5c133de546 None None] Enqueued Resource ID: 08d6f66d-865d-4e89-8403-36be2f89d224 with Priority: HIGHEST for action port
[  537.605206] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08d6f66d-865d-4e89-8403-36be2f89d224 with Priority: HIGHEST for action port
[  537.605583] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.605794] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.605908] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.606170] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.606302] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.606506] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-08d6f66d-865d-4e89-8403-36be2f89d224 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.606654] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-08d6f66d-865d-4e89-8403-36be2f89d224" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.606757] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-08d6f66d-865d-4e89-8403-36be2f89d224" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.607028] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-08d6f66d-865d-4e89-8403-36be2f89d224" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.607158] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('08d6f66d-865d-4e89-8403-36be2f89d224',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  537.607391] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a07b12cc89cc429c8b6538a5d45b45aa in queue reply_156db6c4256a4c58884afce15e6f5340
[  537.613937] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a07b12cc89cc429c8b6538a5d45b45aa
[  537.708584] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.708821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.709138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.709470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.709726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.710073] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.710316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.710487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.710815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.711170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.711489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.711820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.712129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.712976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.713213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.713936] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.714893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.715104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.716120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.717355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.717764] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c2c5d0f-c062-4c36-8c85-7601082cbc29 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.719820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.721922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.721922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.721922] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c2c5d0f-c062-4c36-8c85-7601082cbc29',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  537.721922] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4924c215b3b84e8b8e0ea00a6a55c31b in queue reply_156db6c4256a4c58884afce15e6f5340
[  537.730172] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4924c215b3b84e8b8e0ea00a6a55c31b
[  537.902584] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.903083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.903083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.903373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.903567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.903954] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.904206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.904378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.904692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.905055] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.905269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.905433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.905716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.905913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.906125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.906490] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.906701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.906869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.907162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.907376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.907689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.907946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.908130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.908411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.908602] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  537.909066] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg dce26cb88d54449bb8df7727ea931635 in queue reply_156db6c4256a4c58884afce15e6f5340
[  537.922079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.922339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.922496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.922855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.923059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.923450] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.923675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.923829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.924410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.924670] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.924744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.924920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.925154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.925361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.925770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.925940] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.926158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.926328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.926606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.926911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.927224] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.927333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.927506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.927788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.929055] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  537.929055] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 520c2e1797a84e5fa8e10678c83b2433 in queue reply_156db6c4256a4c58884afce15e6f5340
[  537.930574] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dce26cb88d54449bb8df7727ea931635
[  537.934463] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 520c2e1797a84e5fa8e10678c83b2433
[  537.989480] env[60890]: INFO networking_nsxv3.common.synchronization [None req-aeef3b56-5219-4310-a246-828e9d856ca9 None None] Enqueued Resource ID: 4c77b391-c49e-4264-acb3-0aa910cd57af with Priority: HIGHEST for action port
[  537.990164] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 4c77b391-c49e-4264-acb3-0aa910cd57af with Priority: HIGHEST for action port
[  537.990618] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.990882] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.991045] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.991362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.991550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  537.991793] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-4c77b391-c49e-4264-acb3-0aa910cd57af taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  537.992034] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-4c77b391-c49e-4264-acb3-0aa910cd57af" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  537.992195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-4c77b391-c49e-4264-acb3-0aa910cd57af" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  537.992536] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-4c77b391-c49e-4264-acb3-0aa910cd57af" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  537.992802] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4c77b391-c49e-4264-acb3-0aa910cd57af',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  537.993096] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 53fe6f71a94d432797bd9fb89a331b97 in queue reply_156db6c4256a4c58884afce15e6f5340
[  538.014599] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 53fe6f71a94d432797bd9fb89a331b97
[  538.108395] env[60890]: INFO networking_nsxv3.common.synchronization [req-fd28f446-8abc-486d-92de-f9c6c3c33d37 req-258857bf-f46d-471a-aa62-0b411a679f34 tempest-ServersAdminNegativeTestJSON-1199624387 tempest-ServersAdminNegativeTestJSON-1199624387-project-member] Enqueued Resource ID: 807320f7-85fa-40dd-aa93-4f085c5d7eb7 with Priority: HIGHEST for action security_group_members
[  538.108395] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 807320f7-85fa-40dd-aa93-4f085c5d7eb7 with Priority: HIGHEST for action security_group_members
[  538.108395] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.108395] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-807320f7-85fa-40dd-aa93-4f085c5d7eb7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-807320f7-85fa-40dd-aa93-4f085c5d7eb7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-807320f7-85fa-40dd-aa93-4f085c5d7eb7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-807320f7-85fa-40dd-aa93-4f085c5d7eb7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.108395] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.108395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.108395] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('807320f7-85fa-40dd-aa93-4f085c5d7eb7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  538.108395] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 947b129d975d428ba886fb6ff941caa1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  538.110750] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-08d6f66d-865d-4e89-8403-36be2f89d224" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.111161] env[60890]: Traceback (most recent call last):
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  538.111161] env[60890]:     timer()
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  538.111161] env[60890]:     cb(*args, **kw)
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  538.111161] env[60890]:     result = function(*args, **kwargs)
[  538.111161] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  538.111161] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  538.111161] env[60890]:     return method(*args, **kwargs)
[  538.111161] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  538.111161] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  538.111161] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  538.111161] env[60890]:     result = self.transport._send(
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  538.111161] env[60890]:     return self._driver.send(target, ctxt, message,
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  538.111161] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  538.111161] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  538.111161] env[60890]:     raise result
[  538.111161] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.111161] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.143537] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 947b129d975d428ba886fb6ff941caa1
[  538.155683] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.156108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.156345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.156751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.156987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.157400] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.157650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.157848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.158193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.158588] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.158851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.159051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.159379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.159618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.159872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.160295] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.160547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.160745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.161167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.161401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.161786] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.162040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.163244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.163585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.163818] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  538.164209] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bc3284d9e62044f49078518ec3c38a71 in queue reply_156db6c4256a4c58884afce15e6f5340
[  538.169686] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bc3284d9e62044f49078518ec3c38a71
[  538.225499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c2c5d0f-c062-4c36-8c85-7601082cbc29" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.226311] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.226311] env[60890]: ERROR oslo_messaging.rpc.server 
[  538.412941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.413312] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.413312] env[60890]: ERROR oslo_messaging.rpc.server 
[  538.431521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.432080] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.432080] env[60890]: ERROR oslo_messaging.rpc.server 
[  538.495063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-4c77b391-c49e-4264-acb3-0aa910cd57af" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.495264] env[60890]: Traceback (most recent call last):
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  538.495317] env[60890]:     timer()
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  538.495317] env[60890]:     cb(*args, **kw)
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  538.495317] env[60890]:     result = function(*args, **kwargs)
[  538.495317] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  538.495317] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  538.495317] env[60890]:     return method(*args, **kwargs)
[  538.495317] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  538.495317] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  538.495317] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  538.495317] env[60890]:     result = self.transport._send(
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  538.495317] env[60890]:     return self._driver.send(target, ctxt, message,
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  538.495317] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  538.495317] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  538.495317] env[60890]:     raise result
[  538.496064] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.496064] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.609782] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.610123] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.610123] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.610399] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.610536] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.610851] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.611040] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.611160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.611419] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.611562] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.611712] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 807320f7-85fa-40dd-aa93-4f085c5d7eb7 already deleted.
[  538.611866] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-807320f7-85fa-40dd-aa93-4f085c5d7eb7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.671959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.671959] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  538.671959] env[60890]: ERROR oslo_messaging.rpc.server 
[  538.678190] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.678392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.678517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.678796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.678940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.679202] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.679356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.679470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.679853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.679967] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.680140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.680256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.680464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.680610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.680757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.681008] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.681160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.681273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.681478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.681609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  538.681842] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  538.682036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  538.682712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  538.682823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  538.682998] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  538.683250] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ddddf65ff41944b78af89bf75cda3735 in queue reply_156db6c4256a4c58884afce15e6f5340
[  538.694960] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ddddf65ff41944b78af89bf75cda3735
[  539.185114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  539.185639] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  539.185639] env[60890]: ERROR oslo_messaging.rpc.server 
[  539.907403] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.907696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.907798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.908019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.908176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.908440] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.908599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.908718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.908936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.909185] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.909343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.909462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.909670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.909815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.909964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.910215] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.910368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.910485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.910692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.910827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.911628] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.911628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.911628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.911628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.911868] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  539.911998] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg da35c45ce1e1479ab58752b74d512719 in queue reply_156db6c4256a4c58884afce15e6f5340
[  539.929775] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg da35c45ce1e1479ab58752b74d512719
[  539.936612] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.936612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.936782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.937048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.937213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.937478] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.937637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.937754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.938013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.938266] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.938425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.938543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.938757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.938902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.939051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.939303] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.939459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.939625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.939782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.939917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  539.940207] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  539.940390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  539.940444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  539.940654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  539.940790] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  539.941052] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 82263e40ee664d4eabb61ceac2c0df6b in queue reply_156db6c4256a4c58884afce15e6f5340
[  539.948588] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 82263e40ee664d4eabb61ceac2c0df6b
[  540.001348] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.001559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.001686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.001958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.002106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.002365] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.002525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.002709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.002893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.003141] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.003302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.003423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.003637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.003964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.003964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.004196] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.004428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.004484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.004774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.005017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.005289] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.005457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.005594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.005860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.006006] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  540.006254] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bb4fea35f60142a586b0f9ee36238f2f in queue reply_156db6c4256a4c58884afce15e6f5340
[  540.014513] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bb4fea35f60142a586b0f9ee36238f2f
[  540.045148] env[60890]: INFO networking_nsxv3.common.synchronization [None req-b98d79e7-9b91-42ec-8f19-2176430a1092 None None] Enqueued Resource ID: fa17d88b-e138-44dc-af9f-94c9d108de5f with Priority: HIGHEST for action port
[  540.045344] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: fa17d88b-e138-44dc-af9f-94c9d108de5f with Priority: HIGHEST for action port
[  540.045773] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.045836] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.045872] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.046074] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.046158] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.046310] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-fa17d88b-e138-44dc-af9f-94c9d108de5f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.046401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-fa17d88b-e138-44dc-af9f-94c9d108de5f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.046463] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-fa17d88b-e138-44dc-af9f-94c9d108de5f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.046688] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-fa17d88b-e138-44dc-af9f-94c9d108de5f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.046736] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fa17d88b-e138-44dc-af9f-94c9d108de5f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  540.046919] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b9c9425dd85241a59e81604739635015 in queue reply_156db6c4256a4c58884afce15e6f5340
[  540.050264] env[60890]: INFO networking_nsxv3.common.synchronization [req-7aa552ef-de94-4607-83a3-a08f5006a11d req-928770a4-90d0-44ff-acd2-4ebe8da67edc tempest-ServerDiagnosticsTest-269439887 tempest-ServerDiagnosticsTest-269439887-project-member] Enqueued Resource ID: 1bfc53ee-d4bc-45f5-9de4-febc384e1ca2 with Priority: HIGHEST for action security_group_members
[  540.050409] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 1bfc53ee-d4bc-45f5-9de4-febc384e1ca2 with Priority: HIGHEST for action security_group_members
[  540.051609] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.051609] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.051609] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.051796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.051796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.051877] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.051927] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.052083] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.052492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.052492] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.052492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.052492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.052779] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.052779] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.052779] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('1bfc53ee-d4bc-45f5-9de4-febc384e1ca2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  540.052892] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6163d5bebd364830bf4cd1f97e88e91d in queue reply_156db6c4256a4c58884afce15e6f5340
[  540.065215] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b9c9425dd85241a59e81604739635015
[  540.074979] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6163d5bebd364830bf4cd1f97e88e91d
[  540.414869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  540.415405] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  540.415405] env[60890]: ERROR oslo_messaging.rpc.server 
[  540.448152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  540.448152] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  540.448152] env[60890]: ERROR oslo_messaging.rpc.server 
[  540.508214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  540.508733] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  540.508733] env[60890]: ERROR oslo_messaging.rpc.server 
[  540.549398] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-fa17d88b-e138-44dc-af9f-94c9d108de5f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.549654] env[60890]: Traceback (most recent call last):
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  540.549713] env[60890]:     timer()
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  540.549713] env[60890]:     cb(*args, **kw)
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  540.549713] env[60890]:     result = function(*args, **kwargs)
[  540.549713] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  540.549713] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  540.549713] env[60890]:     return method(*args, **kwargs)
[  540.549713] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  540.549713] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  540.549713] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  540.549713] env[60890]:     result = self.transport._send(
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  540.549713] env[60890]:     return self._driver.send(target, ctxt, message,
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  540.549713] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  540.549713] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  540.549713] env[60890]:     raise result
[  540.550357] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  540.550357] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  540.555750] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.555925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.556021] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.556256] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.556346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.556542] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  540.556635] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  540.556694] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  540.556836] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  540.556908] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  540.556997] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 1bfc53ee-d4bc-45f5-9de4-febc384e1ca2 already deleted.
[  540.557137] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-1bfc53ee-d4bc-45f5-9de4-febc384e1ca2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.290279] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.290279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.290279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.290279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.290279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.290741] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.290741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.290741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.290923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.291170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.291342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.291437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.291653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.291800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.291942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.292211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.292386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.292477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.292718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.293744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.293744] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.293744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.293744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.293744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.293744] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  541.294001] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fb194fc573174117a07b3bd670b1ea02 in queue reply_156db6c4256a4c58884afce15e6f5340
[  541.304713] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fb194fc573174117a07b3bd670b1ea02
[  541.328069] env[60890]: INFO networking_nsxv3.common.synchronization [None req-3d2f2ba3-2069-44f7-a013-050d64b5cf43 None None] Enqueued Resource ID: acbfa1ac-d779-4c72-b152-2203c5409451 with Priority: HIGHEST for action port
[  541.328069] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: acbfa1ac-d779-4c72-b152-2203c5409451 with Priority: HIGHEST for action port
[  541.328069] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.330236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.330236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.330236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.330236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.330236] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-acbfa1ac-d779-4c72-b152-2203c5409451 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.330236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-acbfa1ac-d779-4c72-b152-2203c5409451" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.330236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-acbfa1ac-d779-4c72-b152-2203c5409451" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.330236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-acbfa1ac-d779-4c72-b152-2203c5409451" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.330236] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('acbfa1ac-d779-4c72-b152-2203c5409451',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  541.330236] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1008eb2521e54b469a8d3f85f31ebba6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  541.359566] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1008eb2521e54b469a8d3f85f31ebba6
[  541.525192] env[60890]: INFO networking_nsxv3.common.synchronization [req-32b0b0c8-f686-42ae-b571-48f1dc079745 req-97489365-d878-4d30-9a84-6b157a780f62 tempest-ListImageFiltersTestJSON-1201112359 tempest-ListImageFiltersTestJSON-1201112359-project-member] Enqueued Resource ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c with Priority: HIGHEST for action security_group_members
[  541.525342] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c with Priority: HIGHEST for action security_group_members
[  541.525676] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.525781] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.525847] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.526292] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.526292] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.526292] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e0b6278a-c9cf-4011-aaba-b3bfed71204c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.526454] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.526454] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.526616] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.526755] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.526848] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.526906] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.527035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.527377] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.527377] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e0b6278a-c9cf-4011-aaba-b3bfed71204c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  541.527377] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4fae3af8a8e94f46af9b42ed9a75c66e in queue reply_156db6c4256a4c58884afce15e6f5340
[  541.536766] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4fae3af8a8e94f46af9b42ed9a75c66e
[  541.796583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  541.797149] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  541.797149] env[60890]: ERROR oslo_messaging.rpc.server 
[  541.814423] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.814940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.814940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.815132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.815423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.815549] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.815715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.816061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.816061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.816314] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.816806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.816806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.816806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.816951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.817075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.818235] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.818235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.818235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.818235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.818235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.818235] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  541.818806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  541.818806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  541.818806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  541.818806] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  541.819070] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 41ac271a44aa49ff898c7d7408d85f72 in queue reply_156db6c4256a4c58884afce15e6f5340
[  541.830930] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 41ac271a44aa49ff898c7d7408d85f72
[  541.831106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-acbfa1ac-d779-4c72-b152-2203c5409451" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  541.831323] env[60890]: Traceback (most recent call last):
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  541.831377] env[60890]:     timer()
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  541.831377] env[60890]:     cb(*args, **kw)
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  541.831377] env[60890]:     result = function(*args, **kwargs)
[  541.831377] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  541.831377] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  541.831377] env[60890]:     return method(*args, **kwargs)
[  541.831377] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  541.831377] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  541.831377] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  541.831377] env[60890]:     result = self.transport._send(
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  541.831377] env[60890]:     return self._driver.send(target, ctxt, message,
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  541.831377] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  541.831377] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  541.831377] env[60890]:     raise result
[  541.831377] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  541.831377] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.012220] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.012220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.012220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.012220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.012220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.012220] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.012220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.012220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.012850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.012850] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.012850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.013034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.013115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.013228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.013383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.013633] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.013790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.014286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.014543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.014762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.015196] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.015444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.015648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.015872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.016071] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  542.016347] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8fa67f10de0442e28db469a14792b1ba in queue reply_156db6c4256a4c58884afce15e6f5340
[  542.018312] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.018482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.018608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.018829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.018971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.019216] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.019373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.019639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.019718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.020030] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.020246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.020400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.020642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.020814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.020997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.022191] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.022191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.022191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.022191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.022460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.022604] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.022889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.023068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.024145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.024449] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  542.024760] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f0dada70dde941c385ea1d598e8c1e76 in queue reply_156db6c4256a4c58884afce15e6f5340
[  542.025854] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8fa67f10de0442e28db469a14792b1ba
[  542.026501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.027002] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.027002] env[60890]: ERROR oslo_messaging.rpc.server 
[  542.029512] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.029630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.029701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.029865] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.029951] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.030123] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.030222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.030284] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.030427] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.030503] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.030625] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c already deleted.
[  542.030657] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.041037] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f0dada70dde941c385ea1d598e8c1e76
[  542.321105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.321635] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.321635] env[60890]: ERROR oslo_messaging.rpc.server 
[  542.458031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.458249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.458380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.458652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.458797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.459060] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.459219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.459339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.459555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.459820] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.459944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.460078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.460355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.460563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.460563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.460825] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.461008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.461298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.461375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.461473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.461713] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.461869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.461981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.462192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.462331] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  542.463404] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0a5d6fbdee44494587cfa0376eb80d7b in queue reply_156db6c4256a4c58884afce15e6f5340
[  542.469764] env[60890]: INFO networking_nsxv3.common.synchronization [None req-c688b068-4955-4a63-9753-4e94d9559130 None None] Enqueued Resource ID: ffa9a24c-ccd7-4a98-8be0-28bf9ddd4b65 with Priority: HIGHEST for action port
[  542.470345] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0a5d6fbdee44494587cfa0376eb80d7b
[  542.470943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.471371] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.471371] env[60890]: ERROR oslo_messaging.rpc.server 
[  542.472409] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffa9a24c-ccd7-4a98-8be0-28bf9ddd4b65 with Priority: HIGHEST for action port
[  542.473229] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.473378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.473430] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.473598] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.473679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.473827] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-ffa9a24c-ccd7-4a98-8be0-28bf9ddd4b65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.474033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-ffa9a24c-ccd7-4a98-8be0-28bf9ddd4b65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.474100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-ffa9a24c-ccd7-4a98-8be0-28bf9ddd4b65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.474306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-ffa9a24c-ccd7-4a98-8be0-28bf9ddd4b65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.474381] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ffa9a24c-ccd7-4a98-8be0-28bf9ddd4b65',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  542.474620] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 21a925ea5079474faa2a827e233a90b9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  542.479994] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 21a925ea5079474faa2a827e233a90b9
[  542.532642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.533170] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  542.533170] env[60890]: ERROR oslo_messaging.rpc.server 
[  542.600541] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.600747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.600873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.601141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.601282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.601540] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.601723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.601807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.602027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.602276] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.602433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.602554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.602792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.602940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.603089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.603339] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.603488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.603602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.603807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.603938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.604198] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.604337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.604455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.604676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.604825] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  542.605052] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3e0f1fe0d56f46419cd33b886e5a5299 in queue reply_156db6c4256a4c58884afce15e6f5340
[  542.614792] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3e0f1fe0d56f46419cd33b886e5a5299
[  542.909894] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.910094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.910222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.910491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.910638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.910893] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.911052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.911168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.911386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.911627] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.911782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.911899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.912120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.912264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.912409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.912686] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.912844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.913223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.913484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.913628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.913873] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  542.914030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  542.914189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  542.916171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  542.916171] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  542.916171] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 474f1e18af5e4711aa242b870c4464d4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  542.921925] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 474f1e18af5e4711aa242b870c4464d4
[  542.977264] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-ffa9a24c-ccd7-4a98-8be0-28bf9ddd4b65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  542.977264] env[60890]: Traceback (most recent call last):
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  542.977264] env[60890]:     timer()
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  542.977264] env[60890]:     cb(*args, **kw)
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  542.977264] env[60890]:     result = function(*args, **kwargs)
[  542.977264] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  542.977264] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  542.977264] env[60890]:     return method(*args, **kwargs)
[  542.977264] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  542.977264] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  542.977264] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  542.977264] env[60890]:     result = self.transport._send(
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  542.977264] env[60890]:     return self._driver.send(target, ctxt, message,
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  542.977264] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  542.977264] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  542.977264] env[60890]:     raise result
[  542.977264] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  542.977264] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.115535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  543.115535] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.115535] env[60890]: ERROR oslo_messaging.rpc.server 
[  543.127823] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.128041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.128175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.128444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.128577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.128837] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.128997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.129115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.129327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.129570] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.129727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.129846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.130053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.130229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.130337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.130588] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.130744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.130866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.131074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.131209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.131448] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-77c53c68-7546-4376-b886-867e8cee09e3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.131602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.131720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.131947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.132123] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('77c53c68-7546-4376-b886-867e8cee09e3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  543.132361] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2d5b86da46584f358f8a307c8ada2cdd in queue reply_156db6c4256a4c58884afce15e6f5340
[  543.147897] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2d5b86da46584f358f8a307c8ada2cdd
[  543.245106] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.245106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.245106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.245406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.245406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.246049] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.246049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.246049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.246222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.246402] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.246551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.246663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.246867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.247033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.247201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.247448] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.247598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.247729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.249266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.249266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.249266] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a740d346-3512-47aa-a8a1-637ada8690a5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.249266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.249266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.249721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.249915] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a740d346-3512-47aa-a8a1-637ada8690a5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  543.250191] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5f8235a755cf4121af7140b579cda895 in queue reply_156db6c4256a4c58884afce15e6f5340
[  543.258812] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5f8235a755cf4121af7140b579cda895
[  543.323548] env[60890]: INFO networking_nsxv3.common.synchronization [None req-70cc6eed-fe3a-44d3-a20b-8b3cfaf1277c tempest-ServersTestBootFromVolume-1482275162 tempest-ServersTestBootFromVolume-1482275162-project-member] Enqueued Resource ID: e0852405-c71e-4825-ac71-a12a84643215 with Priority: HIGHEST for action security_group_rules
[  543.324090] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e0852405-c71e-4825-ac71-a12a84643215 with Priority: HIGHEST for action security_group_rules
[  543.324419] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.324540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.324605] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.324835] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.324917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.325069] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-e0852405-c71e-4825-ac71-a12a84643215 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.325163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.325223] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.325427] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.325504] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('e0852405-c71e-4825-ac71-a12a84643215',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  543.325700] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a8a30e0c4eb74c9698744c08cd3af4fc in queue reply_156db6c4256a4c58884afce15e6f5340
[  543.357954] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a8a30e0c4eb74c9698744c08cd3af4fc
[  543.417183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  543.418026] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.418026] env[60890]: ERROR oslo_messaging.rpc.server 
[  543.458754] env[60890]: INFO networking_nsxv3.common.synchronization [None req-c54ed7ab-a0b5-4f38-aa6a-ba16ef7f3b6e tempest-ServersTestBootFromVolume-1482275162 tempest-ServersTestBootFromVolume-1482275162-project-member] Enqueued Resource ID: e0852405-c71e-4825-ac71-a12a84643215 with Priority: HIGHEST for action security_group_rules
[  543.459456] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e0852405-c71e-4825-ac71-a12a84643215 with Priority: HIGHEST for action security_group_rules
[  543.460417] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.460622] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.460758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.461070] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.461225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.461489] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-e0852405-c71e-4825-ac71-a12a84643215 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.461636] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.635623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-77c53c68-7546-4376-b886-867e8cee09e3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  543.636225] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.636225] env[60890]: ERROR oslo_messaging.rpc.server 
[  543.713729] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.713729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.713729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.714414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.714414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.714414] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.714414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.714565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.714746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.715027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.715142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.715262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.715484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.716779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.717023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.717295] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.717462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.717587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.717823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.717990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.718241] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.718405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.718574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.719125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.719125] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  543.719125] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c0c0aa5d61824b6ea252c952ba03398d in queue reply_156db6c4256a4c58884afce15e6f5340
[  543.732228] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c0c0aa5d61824b6ea252c952ba03398d
[  543.757272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a740d346-3512-47aa-a8a1-637ada8690a5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  543.757775] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  543.757775] env[60890]: ERROR oslo_messaging.rpc.server 
[  543.845964] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.846124] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.846198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.846414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.846535] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.846609] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: e0852405-c71e-4825-ac71-a12a84643215 already deleted.
[  543.846693] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.846845] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.847031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.847152] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('e0852405-c71e-4825-ac71-a12a84643215',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  543.847311] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3679ac446b9d4253a6f34aee87f3f9db in queue reply_156db6c4256a4c58884afce15e6f5340
[  543.913727] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3679ac446b9d4253a6f34aee87f3f9db
[  543.941999] env[60890]: INFO networking_nsxv3.common.synchronization [None req-f66bdeae-ca23-4b6f-a0a0-1fc08ce58e77 None None] Enqueued Resource ID: db989a39-94da-4c0a-be5a-758b6a24f1d3 with Priority: HIGHEST for action port
[  543.942639] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: db989a39-94da-4c0a-be5a-758b6a24f1d3 with Priority: HIGHEST for action port
[  543.943004] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.943169] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.943271] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.943508] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.943648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  543.943953] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-db989a39-94da-4c0a-be5a-758b6a24f1d3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  543.943953] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-db989a39-94da-4c0a-be5a-758b6a24f1d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  543.943953] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-db989a39-94da-4c0a-be5a-758b6a24f1d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  543.944254] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-db989a39-94da-4c0a-be5a-758b6a24f1d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  543.944357] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('db989a39-94da-4c0a-be5a-758b6a24f1d3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  543.944564] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7bb7c4618e754fbe91b3451b5b275f1f in queue reply_156db6c4256a4c58884afce15e6f5340
[  543.976668] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7bb7c4618e754fbe91b3451b5b275f1f
[  544.135743] env[60890]: INFO networking_nsxv3.common.synchronization [None req-3dd2fc06-675d-4d5b-bbba-4324e672cce2 None None] Enqueued Resource ID: 35133c3e-1083-485a-8387-1dff399d8f9a with Priority: HIGHEST for action port
[  544.136010] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 35133c3e-1083-485a-8387-1dff399d8f9a with Priority: HIGHEST for action port
[  544.136609] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.136735] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.136803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.137004] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.137087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.137232] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-35133c3e-1083-485a-8387-1dff399d8f9a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.137326] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-35133c3e-1083-485a-8387-1dff399d8f9a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.137380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-35133c3e-1083-485a-8387-1dff399d8f9a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.137581] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-35133c3e-1083-485a-8387-1dff399d8f9a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.137661] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('35133c3e-1083-485a-8387-1dff399d8f9a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  544.137842] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9a7d08bfad9e47d5a0afda9a9f0e0e4f in queue reply_156db6c4256a4c58884afce15e6f5340
[  544.153425] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9a7d08bfad9e47d5a0afda9a9f0e0e4f
[  544.221111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  544.221745] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  544.221745] env[60890]: ERROR oslo_messaging.rpc.server 
[  544.237438] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.237438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.237438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.237438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.237438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.237438] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.237438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.237438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.237438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.237438] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.237438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.237943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.237943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.237943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.238075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.238330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.238476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.238588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.238802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.238969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.239896] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.240197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.240401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.240676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.240900] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  544.241229] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d2d05687cd434d33975e6b5a2d1d95db in queue reply_156db6c4256a4c58884afce15e6f5340
[  544.265114] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d2d05687cd434d33975e6b5a2d1d95db
[  544.368541] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.368541] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.368541] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.368541] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.368541] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.368541] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: e0852405-c71e-4825-ac71-a12a84643215 already deleted.
[  544.368541] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.446614] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-db989a39-94da-4c0a-be5a-758b6a24f1d3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.446614] env[60890]: Traceback (most recent call last):
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  544.446614] env[60890]:     timer()
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  544.446614] env[60890]:     cb(*args, **kw)
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  544.446614] env[60890]:     result = function(*args, **kwargs)
[  544.446614] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  544.446614] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  544.446614] env[60890]:     return method(*args, **kwargs)
[  544.446614] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  544.446614] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  544.446614] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  544.446614] env[60890]:     result = self.transport._send(
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  544.446614] env[60890]:     return self._driver.send(target, ctxt, message,
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  544.446614] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  544.446614] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  544.446614] env[60890]:     raise result
[  544.446614] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  544.446614] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  544.641517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-35133c3e-1083-485a-8387-1dff399d8f9a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.641517] env[60890]: Traceback (most recent call last):
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  544.641517] env[60890]:     timer()
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  544.641517] env[60890]:     cb(*args, **kw)
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  544.641517] env[60890]:     result = function(*args, **kwargs)
[  544.641517] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  544.641517] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  544.641517] env[60890]:     return method(*args, **kwargs)
[  544.641517] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  544.641517] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  544.641517] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  544.641517] env[60890]:     result = self.transport._send(
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  544.641517] env[60890]:     return self._driver.send(target, ctxt, message,
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  544.641517] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  544.641517] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  544.641517] env[60890]:     raise result
[  544.641517] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  544.641517] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  544.746649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  544.747199] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  544.747199] env[60890]: ERROR oslo_messaging.rpc.server 
[  544.777934] env[60890]: INFO networking_nsxv3.common.synchronization [None req-9ee69770-c6c1-4f92-b7ee-e4b82643f07f None None] Enqueued Resource ID: c296c348-d550-4e7c-99f3-dfe6ad0dc6d1 with Priority: HIGHEST for action port
[  544.778452] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c296c348-d550-4e7c-99f3-dfe6ad0dc6d1 with Priority: HIGHEST for action port
[  544.778773] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.778929] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.779009] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.779207] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.779471] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  544.779471] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-c296c348-d550-4e7c-99f3-dfe6ad0dc6d1 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  544.779608] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-c296c348-d550-4e7c-99f3-dfe6ad0dc6d1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  544.779608] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-c296c348-d550-4e7c-99f3-dfe6ad0dc6d1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  544.779791] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-c296c348-d550-4e7c-99f3-dfe6ad0dc6d1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  544.779866] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c296c348-d550-4e7c-99f3-dfe6ad0dc6d1',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  544.780063] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 373ce18e843f436ab1c1dcf0b5b3058b in queue reply_156db6c4256a4c58884afce15e6f5340
[  544.830437] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 373ce18e843f436ab1c1dcf0b5b3058b
[  545.128467] env[60890]: INFO networking_nsxv3.common.synchronization [req-484b83ed-8792-43ac-9941-0470349490c7 req-409918cd-c2fd-433f-96d1-624b50576bb5 tempest-TenantUsagesTestJSON-1613839861 tempest-TenantUsagesTestJSON-1613839861-project-member] Enqueued Resource ID: 27022d8e-fbf0-46b2-bdab-d93cd9cc0c46 with Priority: HIGHEST for action security_group_members
[  545.128998] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 27022d8e-fbf0-46b2-bdab-d93cd9cc0c46 with Priority: HIGHEST for action security_group_members
[  545.132679] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.132679] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.132679] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.132679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.132679] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('27022d8e-fbf0-46b2-bdab-d93cd9cc0c46',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  545.132679] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8a25c5608e2844f085e78f96948b3e69 in queue reply_156db6c4256a4c58884afce15e6f5340
[  545.145282] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8a25c5608e2844f085e78f96948b3e69
[  545.200290] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.200486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.200608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.200895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.201037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.201285] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-501 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.201513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.201570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.201758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.201996] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.202143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.202256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.202570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.202623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.202751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-501" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.202997] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.203145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.203281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.203473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.203602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.203845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-96658bde-5d16-4e7a-ab56-d276e8b33d69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.203993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.204120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.204329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.204451] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('96658bde-5d16-4e7a-ab56-d276e8b33d69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  545.204718] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a12fbee166ba4203b8f394eea6fad587 in queue reply_156db6c4256a4c58884afce15e6f5340
[  545.211231] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a12fbee166ba4203b8f394eea6fad587
[  545.255042] env[60890]: INFO networking_nsxv3.common.synchronization [req-35a4c6f8-720e-429c-8df5-13f67850aad2 req-04e4fa57-d7a3-447e-a136-9e3db9e61539 tempest-VolumesAdminNegativeTest-2086354477 tempest-VolumesAdminNegativeTest-2086354477-project-member] Enqueued Resource ID: b371c264-efdf-44ab-9a75-9ff23ee9391e with Priority: HIGHEST for action security_group_members
[  545.255615] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b371c264-efdf-44ab-9a75-9ff23ee9391e with Priority: HIGHEST for action security_group_members
[  545.255943] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.256095] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.256167] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.263378] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b371c264-efdf-44ab-9a75-9ff23ee9391e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.263378] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.263378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.263378] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b371c264-efdf-44ab-9a75-9ff23ee9391e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  545.263378] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 23d1dc8a57d848019a76504a249cf2e1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  545.276990] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 23d1dc8a57d848019a76504a249cf2e1
[  545.281521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-c296c348-d550-4e7c-99f3-dfe6ad0dc6d1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.281734] env[60890]: Traceback (most recent call last):
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  545.281781] env[60890]:     timer()
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  545.281781] env[60890]:     cb(*args, **kw)
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  545.281781] env[60890]:     result = function(*args, **kwargs)
[  545.281781] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  545.281781] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  545.281781] env[60890]:     return method(*args, **kwargs)
[  545.281781] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  545.281781] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  545.281781] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  545.281781] env[60890]:     result = self.transport._send(
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  545.281781] env[60890]:     return self._driver.send(target, ctxt, message,
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  545.281781] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  545.281781] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  545.281781] env[60890]:     raise result
[  545.282530] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  545.282530] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  545.389303] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.389598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.389647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.389881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.390023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.390296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.390456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.390577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.390789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.391027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.391178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.391294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.391498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.391637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.391782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.392042] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.392198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.392322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.392533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.392673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.392918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.393072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.393192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.393402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.393539] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  545.393775] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 699dd83b0fa8401680e8ffa71d7e406a in queue reply_156db6c4256a4c58884afce15e6f5340
[  545.408472] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 699dd83b0fa8401680e8ffa71d7e406a
[  545.632819] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.632989] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.633042] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.633242] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.633337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.633532] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.633634] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.633697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.633855] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.633926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.634011] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 27022d8e-fbf0-46b2-bdab-d93cd9cc0c46 already deleted.
[  545.634092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-27022d8e-fbf0-46b2-bdab-d93cd9cc0c46" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.707334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-96658bde-5d16-4e7a-ab56-d276e8b33d69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  545.707894] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  545.707894] env[60890]: ERROR oslo_messaging.rpc.server 
[  545.764945] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.765067] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.765154] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.771035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.771035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.771035] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.771035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.771035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.771035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.771035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.771035] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b371c264-efdf-44ab-9a75-9ff23ee9391e already deleted.
[  545.771035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b371c264-efdf-44ab-9a75-9ff23ee9391e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.898829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  545.898829] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  545.898829] env[60890]: ERROR oslo_messaging.rpc.server 
[  545.972276] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.972486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.972615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.972894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.973043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.973302] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.973461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.973580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.973800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.974045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.974229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.974315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.974529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.974687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.974877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.975137] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.975484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.975636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.975897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.976092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  545.976292] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  545.976445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  545.976585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  545.976767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  545.976899] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  545.977195] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg edc50d1f32ff4cf4a34ce5d329081fc9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  545.982148] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg edc50d1f32ff4cf4a34ce5d329081fc9
[  546.484652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  546.485230] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  546.485230] env[60890]: ERROR oslo_messaging.rpc.server 
[  547.009655] env[60890]: INFO networking_nsxv3.common.synchronization [req-ed00e15f-3dc4-47d2-bf0a-eee7d7714ed5 req-adfc47bd-86b7-408c-86a5-c1218f327975 tempest-ImagesNegativeTestJSON-494836937 tempest-ImagesNegativeTestJSON-494836937-project-member] Enqueued Resource ID: 7663847a-5074-4a6e-81cc-c7458ccf0f4c with Priority: HIGHEST for action security_group_members
[  547.010346] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7663847a-5074-4a6e-81cc-c7458ccf0f4c with Priority: HIGHEST for action security_group_members
[  547.010661] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.010806] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.010878] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.011075] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.011158] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.011307] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7663847a-5074-4a6e-81cc-c7458ccf0f4c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.011400] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7663847a-5074-4a6e-81cc-c7458ccf0f4c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.011459] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7663847a-5074-4a6e-81cc-c7458ccf0f4c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.011604] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7663847a-5074-4a6e-81cc-c7458ccf0f4c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.011740] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.011826] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.011883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.012033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.012113] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.012192] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7663847a-5074-4a6e-81cc-c7458ccf0f4c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  547.012382] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 96e8b04e0b9e405384a0a34ad3ed1ba1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  547.022064] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 96e8b04e0b9e405384a0a34ad3ed1ba1
[  547.218336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.218548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.218687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.219014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.219132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.219346] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.219495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.219615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.219827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.220086] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.220244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.220359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.220563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.220707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.220861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.221232] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.221411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.221530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.221739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.221865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.222111] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.222264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.222875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.222875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.222875] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  547.223039] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 646384f4d8d04837a772c3f519ec3118 in queue reply_156db6c4256a4c58884afce15e6f5340
[  547.230337] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 646384f4d8d04837a772c3f519ec3118
[  547.520475] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.520812] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.520863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.521071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.521165] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.521354] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.521456] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.521512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.521668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.521750] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.521836] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7663847a-5074-4a6e-81cc-c7458ccf0f4c already deleted.
[  547.521915] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7663847a-5074-4a6e-81cc-c7458ccf0f4c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.725924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  547.725999] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  547.725999] env[60890]: ERROR oslo_messaging.rpc.server 
[  547.788711] env[60890]: INFO networking_nsxv3.common.synchronization [None req-72efe27d-5ef7-4e80-9b7a-4cb937ef6bb2 None None] Enqueued Resource ID: c52f6c91-81bb-4d1f-be62-0abbe340bea8 with Priority: HIGHEST for action port
[  547.788987] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c52f6c91-81bb-4d1f-be62-0abbe340bea8 with Priority: HIGHEST for action port
[  547.789539] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.789675] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.789740] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.789945] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.790038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  547.790191] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-c52f6c91-81bb-4d1f-be62-0abbe340bea8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  547.790286] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-c52f6c91-81bb-4d1f-be62-0abbe340bea8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  547.790375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-c52f6c91-81bb-4d1f-be62-0abbe340bea8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  547.790560] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-c52f6c91-81bb-4d1f-be62-0abbe340bea8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  547.790663] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c52f6c91-81bb-4d1f-be62-0abbe340bea8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  547.790809] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg eaedb2ed8c9e459da1be0e65e42d4667 in queue reply_156db6c4256a4c58884afce15e6f5340
[  547.802579] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eaedb2ed8c9e459da1be0e65e42d4667
[  548.292733] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-c52f6c91-81bb-4d1f-be62-0abbe340bea8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  548.292986] env[60890]: Traceback (most recent call last):
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  548.293035] env[60890]:     timer()
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  548.293035] env[60890]:     cb(*args, **kw)
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  548.293035] env[60890]:     result = function(*args, **kwargs)
[  548.293035] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  548.293035] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  548.293035] env[60890]:     return method(*args, **kwargs)
[  548.293035] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  548.293035] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  548.293035] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  548.293035] env[60890]:     result = self.transport._send(
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  548.293035] env[60890]:     return self._driver.send(target, ctxt, message,
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  548.293035] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  548.293035] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  548.293035] env[60890]:     raise result
[  548.293768] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  548.293768] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  548.436100] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  548.436100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  548.436355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  548.436605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  548.436605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  548.437156] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  548.437156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  548.437372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  548.437609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  548.437842] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  548.437963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  548.438083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  548.438363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  548.438527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  548.438687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  548.438949] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  548.439177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  548.439237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  548.439500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  548.439641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  548.439895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  548.444794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  548.444794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  548.444794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  548.445342] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  548.445413] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 66ab428de068435b8a2492877da13b36 in queue reply_156db6c4256a4c58884afce15e6f5340
[  548.452454] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 66ab428de068435b8a2492877da13b36
[  548.952430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  548.952430] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  548.952430] env[60890]: ERROR oslo_messaging.rpc.server 
[  549.525779] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  549.525779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  549.525779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  549.525779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  549.525988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  549.526081] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  549.526239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  549.526357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  549.526577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  549.526823] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  549.526985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  549.527106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  549.527324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  549.527466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  549.527671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  549.527854] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  549.528654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  549.528845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  549.529117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  549.529317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  549.529845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  549.530072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  549.530243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  549.530539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  549.530721] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  549.531011] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 666103347a1a479996a473d170a79c6e in queue reply_156db6c4256a4c58884afce15e6f5340
[  549.537280] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 666103347a1a479996a473d170a79c6e
[  550.037456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  550.037456] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  550.037456] env[60890]: ERROR oslo_messaging.rpc.server 
[  550.159620] env[60890]: INFO networking_nsxv3.common.synchronization [None req-4ce86096-a781-4860-9eb0-30e474433244 None None] Enqueued Resource ID: 6143f91a-ff65-46b6-a940-3ec92cdaa038 with Priority: HIGHEST for action port
[  550.159620] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6143f91a-ff65-46b6-a940-3ec92cdaa038 with Priority: HIGHEST for action port
[  550.159620] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  550.159620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  550.159620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  550.159620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  550.159620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  550.159620] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-6143f91a-ff65-46b6-a940-3ec92cdaa038 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  550.159620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-6143f91a-ff65-46b6-a940-3ec92cdaa038" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  550.159620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-6143f91a-ff65-46b6-a940-3ec92cdaa038" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  550.159620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-6143f91a-ff65-46b6-a940-3ec92cdaa038" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  550.159620] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6143f91a-ff65-46b6-a940-3ec92cdaa038',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  550.159620] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f85d7f5197e54be2a9d5a0143cdf146f in queue reply_156db6c4256a4c58884afce15e6f5340
[  550.173985] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f85d7f5197e54be2a9d5a0143cdf146f
[  550.407934] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  550.408183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  550.408347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  550.408739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  550.409008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  550.409349] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  550.409522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  550.409646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  550.409887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  550.410134] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  550.410288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  550.410407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  550.410623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  550.410774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  550.411165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  550.411236] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  550.411393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  550.411517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  550.411738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  550.411902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  550.412359] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  550.412533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  550.412656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  550.412875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  550.413017] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  550.413258] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 962da9f2e3da4daf8f87ee90a3548e99 in queue reply_156db6c4256a4c58884afce15e6f5340
[  550.434074] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 962da9f2e3da4daf8f87ee90a3548e99
[  550.662470] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-6143f91a-ff65-46b6-a940-3ec92cdaa038" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  550.662811] env[60890]: Traceback (most recent call last):
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  550.662811] env[60890]:     timer()
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  550.662811] env[60890]:     cb(*args, **kw)
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  550.662811] env[60890]:     result = function(*args, **kwargs)
[  550.662811] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  550.662811] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  550.662811] env[60890]:     return method(*args, **kwargs)
[  550.662811] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  550.662811] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  550.662811] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  550.662811] env[60890]:     result = self.transport._send(
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  550.662811] env[60890]:     return self._driver.send(target, ctxt, message,
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  550.662811] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  550.662811] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  550.662811] env[60890]:     raise result
[  550.662811] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  550.662811] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  550.915080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  550.915679] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  550.915679] env[60890]: ERROR oslo_messaging.rpc.server 
[  551.460189] env[60890]: INFO networking_nsxv3.common.synchronization [None req-182496d8-dc9b-43e3-bf72-d676edc5a950 None None] Enqueued Resource ID: cc3cbed9-1943-4c20-8c2e-87ddd6e7a84c with Priority: HIGHEST for action port
[  551.460438] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: cc3cbed9-1943-4c20-8c2e-87ddd6e7a84c with Priority: HIGHEST for action port
[  551.460997] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  551.461063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  551.461160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  551.461310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  551.461435] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  551.461581] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-cc3cbed9-1943-4c20-8c2e-87ddd6e7a84c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  551.461644] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-cc3cbed9-1943-4c20-8c2e-87ddd6e7a84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  551.461739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-cc3cbed9-1943-4c20-8c2e-87ddd6e7a84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  551.462598] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-cc3cbed9-1943-4c20-8c2e-87ddd6e7a84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  551.462598] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc3cbed9-1943-4c20-8c2e-87ddd6e7a84c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  551.462598] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7c3e4c36c8e44daaa0e8edd0710b49ea in queue reply_156db6c4256a4c58884afce15e6f5340
[  551.485671] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7c3e4c36c8e44daaa0e8edd0710b49ea
[  551.628194] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  551.628417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  551.628540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  551.628816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  551.629014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  551.629299] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  551.629414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  551.629486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  551.629812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  551.629945] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  551.630095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  551.630207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  551.630439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  551.630579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  551.630742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  551.630983] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  551.631134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  551.631246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  551.631455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  551.631584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  551.632891] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  551.632891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  551.632891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  551.632891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  551.632891] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  551.632891] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ff6d7447bb5f4ed59e91172dc9fc78c4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  551.681688] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ff6d7447bb5f4ed59e91172dc9fc78c4
[  551.965597] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-cc3cbed9-1943-4c20-8c2e-87ddd6e7a84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  551.965883] env[60890]: Traceback (most recent call last):
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  551.965962] env[60890]:     timer()
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  551.965962] env[60890]:     cb(*args, **kw)
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  551.965962] env[60890]:     result = function(*args, **kwargs)
[  551.965962] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  551.965962] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  551.965962] env[60890]:     return method(*args, **kwargs)
[  551.965962] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  551.965962] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  551.965962] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  551.965962] env[60890]:     result = self.transport._send(
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  551.965962] env[60890]:     return self._driver.send(target, ctxt, message,
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  551.965962] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  551.965962] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  551.965962] env[60890]:     raise result
[  551.965962] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  551.965962] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  552.135196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  552.135742] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  552.135742] env[60890]: ERROR oslo_messaging.rpc.server 
[  552.898935] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  552.899236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  552.899464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  552.904119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  552.904119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  552.904119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  552.904119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-dbac6941-decc-4f9f-9424-f0f19c5fd347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  552.904119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  552.904119] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('dbac6941-decc-4f9f-9424-f0f19c5fd347',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  552.904119] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 21d167350ce04ed18233d7eae3477c7f in queue reply_156db6c4256a4c58884afce15e6f5340
[  552.909935] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 21d167350ce04ed18233d7eae3477c7f
[  553.405455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-dbac6941-decc-4f9f-9424-f0f19c5fd347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  553.406825] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  553.406825] env[60890]: ERROR oslo_messaging.rpc.server 
[  554.706725] env[60890]: INFO networking_nsxv3.common.synchronization [None req-e9fad98b-e73e-4f97-8af9-1189a79c389d None None] Enqueued Resource ID: f3fbb23f-73ff-465b-a3c5-a15b2a7a1efd with Priority: HIGHEST for action port
[  554.707236] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f3fbb23f-73ff-465b-a3c5-a15b2a7a1efd with Priority: HIGHEST for action port
[  554.707560] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  554.707703] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  554.707776] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  554.707966] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  554.708059] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  554.708528] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-f3fbb23f-73ff-465b-a3c5-a15b2a7a1efd taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  554.708645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-f3fbb23f-73ff-465b-a3c5-a15b2a7a1efd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  554.708729] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-f3fbb23f-73ff-465b-a3c5-a15b2a7a1efd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  554.708924] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-f3fbb23f-73ff-465b-a3c5-a15b2a7a1efd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  554.709011] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f3fbb23f-73ff-465b-a3c5-a15b2a7a1efd',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  554.709175] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1a98755d78f2435d8d7d97f07decc788 in queue reply_156db6c4256a4c58884afce15e6f5340
[  554.730256] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1a98755d78f2435d8d7d97f07decc788
[  555.118877] env[60890]: INFO networking_nsxv3.common.synchronization [req-32b0b0c8-f686-42ae-b571-48f1dc079745 req-93bd8986-1937-4063-9254-63f8405e1c35 tempest-ListImageFiltersTestJSON-1201112359 tempest-ListImageFiltersTestJSON-1201112359-project-member] Enqueued Resource ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c with Priority: HIGHEST for action security_group_members
[  555.119467] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c with Priority: HIGHEST for action security_group_members
[  555.119822] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  555.119962] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  555.120058] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  555.120287] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  555.120375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  555.120534] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e0b6278a-c9cf-4011-aaba-b3bfed71204c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  555.120635] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  555.120747] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  555.120980] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  555.121149] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  555.121246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  555.121309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  555.121468] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  555.121551] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  555.121646] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e0b6278a-c9cf-4011-aaba-b3bfed71204c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  555.121866] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 560dd48922e34e7a8f58a847cf7d6db2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  555.141105] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 560dd48922e34e7a8f58a847cf7d6db2
[  555.211777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-f3fbb23f-73ff-465b-a3c5-a15b2a7a1efd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  555.212105] env[60890]: Traceback (most recent call last):
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  555.212162] env[60890]:     timer()
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  555.212162] env[60890]:     cb(*args, **kw)
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  555.212162] env[60890]:     result = function(*args, **kwargs)
[  555.212162] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  555.212162] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  555.212162] env[60890]:     return method(*args, **kwargs)
[  555.212162] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  555.212162] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  555.212162] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  555.212162] env[60890]:     result = self.transport._send(
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  555.212162] env[60890]:     return self._driver.send(target, ctxt, message,
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  555.212162] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  555.212162] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  555.212162] env[60890]:     raise result
[  555.212162] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  555.212162] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  555.627944] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  555.627944] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  555.627944] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c already deleted.
[  555.627944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  559.156891] env[60890]: INFO networking_nsxv3.common.synchronization [req-747ba87a-cfaf-471a-a623-86ce852a18e1 req-f28cd8ee-287d-4166-a8d3-b615c9d0c418 tempest-ServerPasswordTestJSON-354071754 tempest-ServerPasswordTestJSON-354071754-project-member] Enqueued Resource ID: 7eee032f-3cc7-4cbf-9083-687ae6ef6ffb with Priority: HIGHEST for action security_group_members
[  559.157457] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7eee032f-3cc7-4cbf-9083-687ae6ef6ffb with Priority: HIGHEST for action security_group_members
[  559.157834] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  559.157948] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  559.157997] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  559.158168] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  559.158263] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  559.158395] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  559.158518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  559.158565] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  559.158806] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  559.159062] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  559.159062] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  559.159199] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  559.159270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  559.159347] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  559.159432] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7eee032f-3cc7-4cbf-9083-687ae6ef6ffb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  559.159615] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 53cdea9e3b83419cb10b67aaa5d69200 in queue reply_156db6c4256a4c58884afce15e6f5340
[  559.188950] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 53cdea9e3b83419cb10b67aaa5d69200
[  559.587683] env[60890]: INFO networking_nsxv3.common.synchronization [None req-31efa86a-192d-4956-a498-947e3cff2d4a None None] Enqueued Resource ID: 52ae3e70-7944-429e-851d-51951d52ac38 with Priority: HIGHEST for action port
[  559.588172] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 52ae3e70-7944-429e-851d-51951d52ac38 with Priority: HIGHEST for action port
[  559.588492] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  559.588621] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  559.588694] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  559.588882] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  559.588957] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  559.589224] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-52ae3e70-7944-429e-851d-51951d52ac38 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  559.589224] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-52ae3e70-7944-429e-851d-51951d52ac38" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  559.589385] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-52ae3e70-7944-429e-851d-51951d52ac38" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  559.589450] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-52ae3e70-7944-429e-851d-51951d52ac38" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  559.589524] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52ae3e70-7944-429e-851d-51951d52ac38',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  559.589697] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 53fe0ffacf574d69b3d9bc8e8ea541aa in queue reply_156db6c4256a4c58884afce15e6f5340
[  559.662050] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  559.662224] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  559.662303] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  559.662568] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  559.662611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  559.662795] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  559.662977] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  559.662977] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  559.663900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  559.663900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  559.663900] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7eee032f-3cc7-4cbf-9083-687ae6ef6ffb already deleted.
[  559.663900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  559.685380] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 53fe0ffacf574d69b3d9bc8e8ea541aa
[  560.091396] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-52ae3e70-7944-429e-851d-51951d52ac38" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  560.091644] env[60890]: Traceback (most recent call last):
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  560.091729] env[60890]:     timer()
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  560.091729] env[60890]:     cb(*args, **kw)
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  560.091729] env[60890]:     result = function(*args, **kwargs)
[  560.091729] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  560.091729] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  560.091729] env[60890]:     return method(*args, **kwargs)
[  560.091729] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  560.091729] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  560.091729] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  560.091729] env[60890]:     result = self.transport._send(
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  560.091729] env[60890]:     return self._driver.send(target, ctxt, message,
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  560.091729] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  560.091729] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  560.091729] env[60890]:     raise result
[  560.091729] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  560.091729] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  561.450945] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  561.451239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  561.451290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  561.451538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  561.451684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  561.452054] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  561.452292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  561.452429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  561.452718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  561.453053] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  561.453225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  561.454433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  561.454433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  561.454508] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  561.454683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  561.454970] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  561.455129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  561.455249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  561.455465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  561.455603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  561.455959] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  561.456151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  561.456274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  561.456534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  561.464230] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  561.464543] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d1e98f3bf575473e9ed45fd9917f5d01 in queue reply_156db6c4256a4c58884afce15e6f5340
[  561.471833] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d1e98f3bf575473e9ed45fd9917f5d01
[  561.472512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  561.473018] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  561.473018] env[60890]: ERROR oslo_messaging.rpc.server 
[  562.614211] env[60890]: INFO networking_nsxv3.common.synchronization [req-7090f4ef-7a3d-442d-a3ef-d4ac45996f56 req-e131952b-f50f-40c2-8cd0-6dc534e18b30 tempest-ImagesTestJSON-598093153 tempest-ImagesTestJSON-598093153-project-member] Enqueued Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  562.614731] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  562.615058] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.615186] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.615253] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.615456] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.615543] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  562.615703] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f0a2229c-87aa-4f25-9157-b1b2e986b23e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.615813] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.615915] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.616128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.616260] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.616355] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.616415] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.616567] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.616649] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  562.616740] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f0a2229c-87aa-4f25-9157-b1b2e986b23e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  562.616927] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9c415a9d2d7446199bcd78e9bebfa6ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  562.629413] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9c415a9d2d7446199bcd78e9bebfa6ad
[  562.728489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.728618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.728747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.729081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.729152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  562.729410] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.729591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.729713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.729945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.730373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.730434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.730546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.730766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.730910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  562.731075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  562.731335] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.731488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.731665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.731865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.731999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  562.732256] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  562.732434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  562.732552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  562.732776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  562.732907] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  562.733187] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5e36853cf61e4655878ba699584da4cd in queue reply_156db6c4256a4c58884afce15e6f5340
[  562.738935] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5e36853cf61e4655878ba699584da4cd
[  563.123157] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  563.123301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  563.123345] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  563.123548] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  563.123647] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  563.123848] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  563.123953] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  563.124052] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  563.124218] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  563.124306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  563.124421] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e already deleted.
[  563.124537] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  563.235536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  563.236129] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  563.236129] env[60890]: ERROR oslo_messaging.rpc.server 
[  563.987079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  563.987079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  563.987079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  563.987079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  563.987079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  563.987079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  563.987079] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  563.987079] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 74753efcb11e4d9eabcd0ac4252526c3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  563.995248] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 74753efcb11e4d9eabcd0ac4252526c3
[  564.443018] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 929ed5e9e9a5440696973c80ea320b26 in queue reply_156db6c4256a4c58884afce15e6f5340
[  564.488320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  564.488846] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  564.488846] env[60890]: ERROR oslo_messaging.rpc.server 
[  564.510445] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 929ed5e9e9a5440696973c80ea320b26
[  564.736544] env[60890]: INFO networking_nsxv3.common.synchronization [None req-3625ecba-e5dd-41a7-b4ac-e8392d0dbe4f None None] Enqueued Resource ID: bbfc526b-b422-453f-8f93-6adcb9246607 with Priority: HIGHEST for action port
[  564.736544] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: bbfc526b-b422-453f-8f93-6adcb9246607 with Priority: HIGHEST for action port
[  564.736544] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  564.736544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  564.736544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  564.736544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  564.736544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  564.736544] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-bbfc526b-b422-453f-8f93-6adcb9246607 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  564.736544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-bbfc526b-b422-453f-8f93-6adcb9246607" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  564.736544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-bbfc526b-b422-453f-8f93-6adcb9246607" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  564.736544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-bbfc526b-b422-453f-8f93-6adcb9246607" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  564.736544] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bbfc526b-b422-453f-8f93-6adcb9246607',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  564.736544] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 628bba71747d4e6b9feeebb6608c3586 in queue reply_156db6c4256a4c58884afce15e6f5340
[  564.773326] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 628bba71747d4e6b9feeebb6608c3586
[  565.192737] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.193001] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.193071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.193344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.193497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.193764] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.193924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.194042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.194261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.194511] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.194803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.194803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.195008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.195153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.195301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.195566] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.195723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.195849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.197116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.197318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.197581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.197747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.197875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.198089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.198226] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  565.198470] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 350859d58d814d5ebe09a80bb00d0afe in queue reply_156db6c4256a4c58884afce15e6f5340
[  565.220946] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 350859d58d814d5ebe09a80bb00d0afe
[  565.238592] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-bbfc526b-b422-453f-8f93-6adcb9246607" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.238732] env[60890]: Traceback (most recent call last):
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  565.238783] env[60890]:     timer()
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  565.238783] env[60890]:     cb(*args, **kw)
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  565.238783] env[60890]:     result = function(*args, **kwargs)
[  565.238783] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  565.238783] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  565.238783] env[60890]:     return method(*args, **kwargs)
[  565.238783] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  565.238783] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  565.238783] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  565.238783] env[60890]:     result = self.transport._send(
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  565.238783] env[60890]:     return self._driver.send(target, ctxt, message,
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  565.238783] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  565.238783] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  565.238783] env[60890]:     raise result
[  565.239604] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  565.239604] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  565.417643] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.417857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.417984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.418252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.418400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.418673] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.418830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.419000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.419281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.419491] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.419655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.419774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.419986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.420144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.420293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.420543] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.420702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.420820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.421027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.421156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.421397] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.421548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.421661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.421911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.422042] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  565.423471] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 91bc3784a8704e34b611ab52f5ba5d77 in queue reply_156db6c4256a4c58884afce15e6f5340
[  565.444320] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 91bc3784a8704e34b611ab52f5ba5d77
[  565.473668] env[60890]: INFO networking_nsxv3.common.synchronization [None req-37fe27f1-a6b2-4e49-9b85-7b153c9ce361 None None] Enqueued Resource ID: 63b131eb-26b8-4a50-941b-944035fe5fde with Priority: HIGHEST for action port
[  565.473668] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 63b131eb-26b8-4a50-941b-944035fe5fde with Priority: HIGHEST for action port
[  565.473668] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.473668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.473668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.473668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.473668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.473668] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-63b131eb-26b8-4a50-941b-944035fe5fde taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  565.473668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-63b131eb-26b8-4a50-941b-944035fe5fde" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  565.473668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-63b131eb-26b8-4a50-941b-944035fe5fde" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  565.473668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-63b131eb-26b8-4a50-941b-944035fe5fde" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  565.473668] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('63b131eb-26b8-4a50-941b-944035fe5fde',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  565.473668] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 651c141584c7462d90e36aeaf0e53293 in queue reply_156db6c4256a4c58884afce15e6f5340
[  565.614319] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 651c141584c7462d90e36aeaf0e53293
[  565.700059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  565.700594] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  565.700594] env[60890]: ERROR oslo_messaging.rpc.server 
[  565.928156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  565.928156] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  565.928156] env[60890]: ERROR oslo_messaging.rpc.server 
[  565.982200] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-63b131eb-26b8-4a50-941b-944035fe5fde" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  565.982447] env[60890]: Traceback (most recent call last):
[  565.982490] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  565.982490] env[60890]:     timer()
[  565.982490] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  565.982490] env[60890]:     cb(*args, **kw)
[  565.982490] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  565.982490] env[60890]:     result = function(*args, **kwargs)
[  565.982490] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  565.982490] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  565.982490] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  565.982490] env[60890]:     return method(*args, **kwargs)
[  565.982490] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  565.982490] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  565.982490] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  565.982490] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  565.982490] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  565.982490] env[60890]:     result = self.transport._send(
[  565.982490] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  565.982490] env[60890]:     return self._driver.send(target, ctxt, message,
[  565.982876] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  565.982876] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  565.982876] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  565.982876] env[60890]:     raise result
[  565.982876] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  565.982876] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  566.125047] env[60890]: INFO networking_nsxv3.common.synchronization [None req-afbc9d1b-bf9a-4f46-b8e2-c7c4aca7a532 None None] Enqueued Resource ID: 79b6bd62-5fe1-42ca-870e-5aa2c2aa6527 with Priority: HIGHEST for action port
[  566.125302] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 79b6bd62-5fe1-42ca-870e-5aa2c2aa6527 with Priority: HIGHEST for action port
[  566.125937] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  566.126062] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  566.126130] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  566.126343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  566.126429] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  566.126582] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-79b6bd62-5fe1-42ca-870e-5aa2c2aa6527 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  566.126680] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-79b6bd62-5fe1-42ca-870e-5aa2c2aa6527" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  566.127120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-79b6bd62-5fe1-42ca-870e-5aa2c2aa6527" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  566.127120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-79b6bd62-5fe1-42ca-870e-5aa2c2aa6527" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  566.127120] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('79b6bd62-5fe1-42ca-870e-5aa2c2aa6527',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  566.127239] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 84ec2ecc7a244c87b0db5fc3adfb69a2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  566.133680] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 84ec2ecc7a244c87b0db5fc3adfb69a2
[  566.629258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-79b6bd62-5fe1-42ca-870e-5aa2c2aa6527" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  566.629610] env[60890]: Traceback (most recent call last):
[  566.629671] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  566.629671] env[60890]:     timer()
[  566.629671] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  566.629671] env[60890]:     cb(*args, **kw)
[  566.629671] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  566.629671] env[60890]:     result = function(*args, **kwargs)
[  566.629671] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  566.629671] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  566.629671] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  566.629671] env[60890]:     return method(*args, **kwargs)
[  566.629671] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  566.629671] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  566.629671] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  566.629671] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  566.629958] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  566.629958] env[60890]:     result = self.transport._send(
[  566.629958] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  566.629958] env[60890]:     return self._driver.send(target, ctxt, message,
[  566.629958] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  566.629958] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  566.629958] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  566.629958] env[60890]:     raise result
[  566.629958] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  566.629958] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  566.800439] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  566.800633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  566.800759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  566.801030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  566.801169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  566.801424] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  566.801578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  566.801702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  566.802023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  566.802279] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  566.802424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  566.802596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  566.802901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  566.803101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  566.803263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  566.803614] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  566.803700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  566.803778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  566.804027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  566.804188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  566.804432] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  566.804587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  566.804783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  566.805000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  566.805165] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  566.805401] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d3e5b3635ed94ac2a901b70b6622099d in queue reply_156db6c4256a4c58884afce15e6f5340
[  566.816166] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d3e5b3635ed94ac2a901b70b6622099d
[  567.202799] env[60890]: INFO networking_nsxv3.common.synchronization [None req-91f6b10a-4e77-4f9b-a664-f2e6775b0837 None None] Enqueued Resource ID: 20d0b5ba-e0ee-43f6-a1a2-ccaff7d96b9f with Priority: HIGHEST for action port
[  567.203030] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 20d0b5ba-e0ee-43f6-a1a2-ccaff7d96b9f with Priority: HIGHEST for action port
[  567.203604] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.203723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.203788] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.204000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.204138] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.204299] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-20d0b5ba-e0ee-43f6-a1a2-ccaff7d96b9f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.204393] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-20d0b5ba-e0ee-43f6-a1a2-ccaff7d96b9f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.204623] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-20d0b5ba-e0ee-43f6-a1a2-ccaff7d96b9f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.204710] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-20d0b5ba-e0ee-43f6-a1a2-ccaff7d96b9f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.204775] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20d0b5ba-e0ee-43f6-a1a2-ccaff7d96b9f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  567.205208] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4f19350248924bd99ad3eb1603c02575 in queue reply_156db6c4256a4c58884afce15e6f5340
[  567.209621] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.209621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.209621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.209621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.209621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.212709] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.212709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.212709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.212709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.212709] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.212962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.212962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.213151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.213531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.213531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.213651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.215712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.215712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.216796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.216796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.216796] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.216796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.216796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.216995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.217564] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  567.218880] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3298fe1723f148019440a4521fc586d8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  567.222933] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4f19350248924bd99ad3eb1603c02575
[  567.226547] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3298fe1723f148019440a4521fc586d8
[  567.308350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  567.308822] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  567.308822] env[60890]: ERROR oslo_messaging.rpc.server 
[  567.720561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-20d0b5ba-e0ee-43f6-a1a2-ccaff7d96b9f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.720816] env[60890]: Traceback (most recent call last):
[  567.720855] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  567.720855] env[60890]:     timer()
[  567.720855] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  567.720855] env[60890]:     cb(*args, **kw)
[  567.720855] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  567.720855] env[60890]:     result = function(*args, **kwargs)
[  567.720855] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  567.720855] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  567.720855] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  567.720855] env[60890]:     return method(*args, **kwargs)
[  567.720855] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  567.720855] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  567.720855] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  567.720855] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  567.721150] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  567.721150] env[60890]:     result = self.transport._send(
[  567.721150] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  567.721150] env[60890]:     return self._driver.send(target, ctxt, message,
[  567.721150] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  567.721150] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  567.721150] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  567.721150] env[60890]:     raise result
[  567.721150] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  567.721150] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  567.724537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  567.725049] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  567.725049] env[60890]: ERROR oslo_messaging.rpc.server 
[  567.960343] env[60890]: INFO networking_nsxv3.common.synchronization [None req-080aad88-ebf0-4bed-af55-5326916d1b5a None None] Enqueued Resource ID: cd0a0c67-2883-4268-9c11-e4097c26b261 with Priority: HIGHEST for action port
[  567.960610] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: cd0a0c67-2883-4268-9c11-e4097c26b261 with Priority: HIGHEST for action port
[  567.961278] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.961408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.961475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.961678] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.961759] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  567.961902] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-cd0a0c67-2883-4268-9c11-e4097c26b261 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  567.961994] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-cd0a0c67-2883-4268-9c11-e4097c26b261" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  567.962048] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-cd0a0c67-2883-4268-9c11-e4097c26b261" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  567.962248] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-cd0a0c67-2883-4268-9c11-e4097c26b261" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  567.962325] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cd0a0c67-2883-4268-9c11-e4097c26b261',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  567.962503] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 041f80941f7f4299bd7f722b640f8ab5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  567.971852] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 041f80941f7f4299bd7f722b640f8ab5
[  568.392186] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.392397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.392528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.392808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.393002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.393275] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.393437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.393558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.393781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.394031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.394189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.394310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.394519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.394696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.394861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.395139] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.395300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.395420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.395687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.395806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.396075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.396236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.396355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.396570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.396704] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  568.396945] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 23c81ce755694413b4fb9b7eff7fadba in queue reply_156db6c4256a4c58884afce15e6f5340
[  568.405059] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 23c81ce755694413b4fb9b7eff7fadba
[  568.465111] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-cd0a0c67-2883-4268-9c11-e4097c26b261" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.465397] env[60890]: Traceback (most recent call last):
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  568.465397] env[60890]:     timer()
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  568.465397] env[60890]:     cb(*args, **kw)
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  568.465397] env[60890]:     result = function(*args, **kwargs)
[  568.465397] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  568.465397] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  568.465397] env[60890]:     return method(*args, **kwargs)
[  568.465397] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  568.465397] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  568.465397] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  568.465397] env[60890]:     result = self.transport._send(
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  568.465397] env[60890]:     return self._driver.send(target, ctxt, message,
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  568.465397] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  568.465397] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  568.465397] env[60890]:     raise result
[  568.465397] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  568.465397] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  568.586560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.586741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.586868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.587176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.587325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.587607] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.588954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.588954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.588954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.588954] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.588954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.588954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.588954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.588954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.589303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.589330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.589475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.589588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.589799] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.589931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.590174] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  568.590327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  568.590443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  568.590654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  568.590792] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  568.591033] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5bfc4784447840818b037b399bc9d179 in queue reply_156db6c4256a4c58884afce15e6f5340
[  568.601917] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5bfc4784447840818b037b399bc9d179
[  568.899091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  568.899619] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  568.899619] env[60890]: ERROR oslo_messaging.rpc.server 
[  569.095434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  569.095996] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  569.095996] env[60890]: ERROR oslo_messaging.rpc.server 
[  569.815245] env[60890]: INFO networking_nsxv3.common.synchronization [None req-049ff61e-8d22-4c45-9e55-3af11fd44927 None None] Enqueued Resource ID: e220372d-684a-400c-b1e1-1ad2ea7ba2fa with Priority: HIGHEST for action port
[  569.815508] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e220372d-684a-400c-b1e1-1ad2ea7ba2fa with Priority: HIGHEST for action port
[  569.816159] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  569.816285] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  569.816350] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  569.816557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  569.816635] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  569.816793] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-e220372d-684a-400c-b1e1-1ad2ea7ba2fa taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  569.816886] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-e220372d-684a-400c-b1e1-1ad2ea7ba2fa" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  569.816944] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-e220372d-684a-400c-b1e1-1ad2ea7ba2fa" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  569.817159] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-e220372d-684a-400c-b1e1-1ad2ea7ba2fa" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  569.817235] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e220372d-684a-400c-b1e1-1ad2ea7ba2fa',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  569.817413] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 85dd24930e394257be43db83d6f48b4f in queue reply_156db6c4256a4c58884afce15e6f5340
[  569.831944] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 85dd24930e394257be43db83d6f48b4f
[  569.887894] env[60890]: INFO networking_nsxv3.common.synchronization [None req-de3cd132-d40c-4342-bd99-9928b09cd2c6 None None] Enqueued Resource ID: 5ad63336-2c89-400f-b806-0452518d6b5a with Priority: HIGHEST for action port
[  569.887894] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5ad63336-2c89-400f-b806-0452518d6b5a with Priority: HIGHEST for action port
[  569.888137] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  569.888265] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  569.890728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  569.890728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  569.890728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  569.890728] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-5ad63336-2c89-400f-b806-0452518d6b5a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  569.890728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-5ad63336-2c89-400f-b806-0452518d6b5a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  569.890728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-5ad63336-2c89-400f-b806-0452518d6b5a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  569.890728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-5ad63336-2c89-400f-b806-0452518d6b5a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  569.890728] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ad63336-2c89-400f-b806-0452518d6b5a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  569.890728] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6bb480d4334540f89f2c44f031a4e4dc in queue reply_156db6c4256a4c58884afce15e6f5340
[  569.896857] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6bb480d4334540f89f2c44f031a4e4dc
[  570.011619] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.011971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.012325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.012732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.012955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.013364] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.013590] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.013761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.014082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.014432] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.014885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.015063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.015352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.015554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.015763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.017071] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.017275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.017400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.017621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.017758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.018007] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.018156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.018283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.018497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.018631] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  570.018867] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f13c23669d7140948a2841b14b4cbeae in queue reply_156db6c4256a4c58884afce15e6f5340
[  570.027751] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f13c23669d7140948a2841b14b4cbeae
[  570.282914] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.283126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.283252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.283530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.283679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.283939] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.284129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.284250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.284473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.284733] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.284895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.285013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.285228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.285373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.285522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.285843] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.286014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.286135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.286350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.286484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.286724] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.286879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.286996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.287202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.287332] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  570.287562] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3557b09fce934b54bee95a2091f8cb97 in queue reply_156db6c4256a4c58884afce15e6f5340
[  570.295777] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3557b09fce934b54bee95a2091f8cb97
[  570.319392] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-e220372d-684a-400c-b1e1-1ad2ea7ba2fa" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.319628] env[60890]: Traceback (most recent call last):
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  570.319676] env[60890]:     timer()
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  570.319676] env[60890]:     cb(*args, **kw)
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  570.319676] env[60890]:     result = function(*args, **kwargs)
[  570.319676] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  570.319676] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  570.319676] env[60890]:     return method(*args, **kwargs)
[  570.319676] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  570.319676] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  570.319676] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  570.319676] env[60890]:     result = self.transport._send(
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  570.319676] env[60890]:     return self._driver.send(target, ctxt, message,
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  570.319676] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  570.319676] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  570.319676] env[60890]:     raise result
[  570.320212] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  570.320212] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  570.394100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-5ad63336-2c89-400f-b806-0452518d6b5a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.394348] env[60890]: Traceback (most recent call last):
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  570.394398] env[60890]:     timer()
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  570.394398] env[60890]:     cb(*args, **kw)
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  570.394398] env[60890]:     result = function(*args, **kwargs)
[  570.394398] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  570.394398] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  570.394398] env[60890]:     return method(*args, **kwargs)
[  570.394398] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  570.394398] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  570.394398] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  570.394398] env[60890]:     result = self.transport._send(
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  570.394398] env[60890]:     return self._driver.send(target, ctxt, message,
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  570.394398] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  570.394398] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  570.394398] env[60890]:     raise result
[  570.394933] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  570.394933] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  570.526126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  570.526126] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  570.526126] env[60890]: ERROR oslo_messaging.rpc.server 
[  570.767608] env[60890]: INFO networking_nsxv3.common.synchronization [None req-fe957bed-2ad7-4dde-95d3-93415d6f9e77 None None] Enqueued Resource ID: 499cd7ba-d602-4535-9d19-a9f633e7237a with Priority: HIGHEST for action port
[  570.767897] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 499cd7ba-d602-4535-9d19-a9f633e7237a with Priority: HIGHEST for action port
[  570.768531] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.768758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.768758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.768915] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.768998] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.769145] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-499cd7ba-d602-4535-9d19-a9f633e7237a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.769237] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-499cd7ba-d602-4535-9d19-a9f633e7237a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.769297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-499cd7ba-d602-4535-9d19-a9f633e7237a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.769503] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-499cd7ba-d602-4535-9d19-a9f633e7237a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.769577] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('499cd7ba-d602-4535-9d19-a9f633e7237a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  570.769769] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0aeb816b91f34f98a9c9e3e5503fd809 in queue reply_156db6c4256a4c58884afce15e6f5340
[  570.782139] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.782139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.782267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.782523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.782676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.782934] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.783089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.783642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.783886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.784190] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.784306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.784421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.784632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.784775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.784921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.785194] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.785345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.785461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.785676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.785807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.786046] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  570.786194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  570.786308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  570.786543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  570.787415] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  570.787665] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 175d3e38a9e540dc9bae14faf831d257 in queue reply_156db6c4256a4c58884afce15e6f5340
[  570.789873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0aeb816b91f34f98a9c9e3e5503fd809
[  570.790599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  570.791048] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  570.791048] env[60890]: ERROR oslo_messaging.rpc.server 
[  570.798576] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 175d3e38a9e540dc9bae14faf831d257
[  571.042790] env[60890]: INFO networking_nsxv3.common.synchronization [req-72ba3ef2-cd37-4d4e-965e-fd5ded9878c1 req-fea3fe50-50cf-4929-bc62-adba90a2fa79 tempest-ListImageFiltersTestJSON-1201112359 tempest-ListImageFiltersTestJSON-1201112359-project-member] Enqueued Resource ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c with Priority: HIGHEST for action security_group_members
[  571.043287] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c with Priority: HIGHEST for action security_group_members
[  571.043634] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.043735] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.043798] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.044040] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.044134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.044286] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e0b6278a-c9cf-4011-aaba-b3bfed71204c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.044402] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.044434] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.044587] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.044776] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.044831] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.044884] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.045036] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.045108] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.045198] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e0b6278a-c9cf-4011-aaba-b3bfed71204c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  571.045371] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1cf28d5b7155474c874c715d7be4174f in queue reply_156db6c4256a4c58884afce15e6f5340
[  571.058363] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1cf28d5b7155474c874c715d7be4174f
[  571.272285] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-499cd7ba-d602-4535-9d19-a9f633e7237a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.272285] env[60890]: Traceback (most recent call last):
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  571.272285] env[60890]:     timer()
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  571.272285] env[60890]:     cb(*args, **kw)
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  571.272285] env[60890]:     result = function(*args, **kwargs)
[  571.272285] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  571.272285] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  571.272285] env[60890]:     return method(*args, **kwargs)
[  571.272285] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  571.272285] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  571.272285] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  571.272285] env[60890]:     result = self.transport._send(
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  571.272285] env[60890]:     return self._driver.send(target, ctxt, message,
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  571.272285] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  571.272285] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  571.272285] env[60890]:     raise result
[  571.272285] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  571.272285] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  571.291959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  571.291959] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  571.291959] env[60890]: ERROR oslo_messaging.rpc.server 
[  571.538783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.539022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.539170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.539484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.539641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.539915] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.540105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.540254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.540610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.540999] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.541255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.541462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.541791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.542055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.542332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.542781] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.543061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.543252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.543492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.543641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.543887] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.549031] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  571.549031] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 577967116e964d3298146bb3067d0df0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  571.549031] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.549031] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.549031] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c already deleted.
[  571.549031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.558313] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 577967116e964d3298146bb3067d0df0
[  571.903260] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.903629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.903842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.904180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.904344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.904623] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.904869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.904988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.905285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.905507] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.905952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.905952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.906047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.906122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.906272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.906518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.906674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.906789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.907036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.907176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  571.907415] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  571.909243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  571.909457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  571.909693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  571.909831] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  571.910073] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 58dd4fbd8bdb4227a8df3f185d4bde22 in queue reply_156db6c4256a4c58884afce15e6f5340
[  571.916427] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 58dd4fbd8bdb4227a8df3f185d4bde22
[  572.048266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  572.048266] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  572.048266] env[60890]: ERROR oslo_messaging.rpc.server 
[  572.413315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  572.413444] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  572.413444] env[60890]: ERROR oslo_messaging.rpc.server 
[  572.470601] env[60890]: INFO networking_nsxv3.common.synchronization [None req-d674bcf5-dffc-4966-9c2a-5c5e280207ad None None] Enqueued Resource ID: 054fb7e7-cc6d-4859-b25c-35f3cf25aa53 with Priority: HIGHEST for action port
[  572.470709] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 054fb7e7-cc6d-4859-b25c-35f3cf25aa53 with Priority: HIGHEST for action port
[  572.471327] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  572.471458] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  572.471526] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  572.471725] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  572.471863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  572.472050] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-054fb7e7-cc6d-4859-b25c-35f3cf25aa53 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  572.472156] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-054fb7e7-cc6d-4859-b25c-35f3cf25aa53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  572.472217] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-054fb7e7-cc6d-4859-b25c-35f3cf25aa53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  572.472427] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-054fb7e7-cc6d-4859-b25c-35f3cf25aa53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  572.472508] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('054fb7e7-cc6d-4859-b25c-35f3cf25aa53',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  572.472699] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e845d9f125d64fee9ae557c9640fb119 in queue reply_156db6c4256a4c58884afce15e6f5340
[  572.477976] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e845d9f125d64fee9ae557c9640fb119
[  572.812673] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  572.812673] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  572.812673] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  572.812673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  572.813558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  572.813735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  572.814031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  572.814167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  572.814290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  572.816117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  572.816117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  572.816117] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  572.816117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  572.816117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  572.816117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  572.816117] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  572.816117] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a8ef8a5db41d46f7a7bcb00c59b6c283 in queue reply_156db6c4256a4c58884afce15e6f5340
[  572.821446] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a8ef8a5db41d46f7a7bcb00c59b6c283
[  572.975515] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-054fb7e7-cc6d-4859-b25c-35f3cf25aa53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  572.975794] env[60890]: Traceback (most recent call last):
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  572.975849] env[60890]:     timer()
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  572.975849] env[60890]:     cb(*args, **kw)
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  572.975849] env[60890]:     result = function(*args, **kwargs)
[  572.975849] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  572.975849] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  572.975849] env[60890]:     return method(*args, **kwargs)
[  572.975849] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  572.975849] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  572.975849] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  572.975849] env[60890]:     result = self.transport._send(
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  572.975849] env[60890]:     return self._driver.send(target, ctxt, message,
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  572.975849] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  572.975849] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  572.975849] env[60890]:     raise result
[  572.976437] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  572.976437] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  573.319367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  573.319894] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  573.319894] env[60890]: ERROR oslo_messaging.rpc.server 
[  573.472047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  573.472047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  573.472047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  573.472047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  573.472047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  573.472047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  573.472047] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  573.472047] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3c0806fa5c67484f8a3c7f9c92f1e392 in queue reply_156db6c4256a4c58884afce15e6f5340
[  573.543434] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3c0806fa5c67484f8a3c7f9c92f1e392
[  573.826477] env[60890]: INFO networking_nsxv3.common.synchronization [None req-e4435ab9-1510-46a7-b556-f256b9aaf315 None None] Enqueued Resource ID: 5673dc28-1fe9-4fd8-9b52-8155e5010a70 with Priority: HIGHEST for action port
[  573.826477] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5673dc28-1fe9-4fd8-9b52-8155e5010a70 with Priority: HIGHEST for action port
[  573.826477] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  573.826477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  573.826477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  573.826477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  573.826477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  573.826477] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-5673dc28-1fe9-4fd8-9b52-8155e5010a70 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  573.826477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-5673dc28-1fe9-4fd8-9b52-8155e5010a70" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  573.826477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-5673dc28-1fe9-4fd8-9b52-8155e5010a70" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  573.826477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-5673dc28-1fe9-4fd8-9b52-8155e5010a70" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  573.826477] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5673dc28-1fe9-4fd8-9b52-8155e5010a70',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  573.827793] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 984eb6e8263b4efaa00662d46c8a14e6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  573.851375] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 984eb6e8263b4efaa00662d46c8a14e6
[  573.985060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  573.987212] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  573.987212] env[60890]: ERROR oslo_messaging.rpc.server 
[  574.088677] env[60890]: INFO networking_nsxv3.common.synchronization [None req-4628de10-e8e3-44ea-8a94-68210b465afa tempest-ServerActionsTestOtherA-363397817 tempest-ServerActionsTestOtherA-363397817-project-member] Enqueued Resource ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f with Priority: HIGHEST for action security_group_rules
[  574.088677] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f with Priority: HIGHEST for action security_group_rules
[  574.088677] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.088677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.088677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.088677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.088677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.088677] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.088677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.088677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.088677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.088677] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('f934cb7a-eebb-4560-bd27-6e115d57ba4f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  574.088677] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b90df50d88914842872eb846c5606933 in queue reply_156db6c4256a4c58884afce15e6f5340
[  574.107397] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b90df50d88914842872eb846c5606933
[  574.164040] env[60890]: INFO networking_nsxv3.common.synchronization [req-288306b3-030f-47a0-9fe8-eb7a4ca729ee req-f6e1f090-85c9-4c02-b616-7b3be15d98c5 tempest-ServerActionsTestOtherB-1179403528 tempest-ServerActionsTestOtherB-1179403528-project-member] Enqueued Resource ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 with Priority: HIGHEST for action security_group_members
[  574.164040] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 with Priority: HIGHEST for action security_group_members
[  574.164040] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.164040] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.164040] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.164319] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.164319] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.164319] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-0cd25894-1483-4857-b8b7-2f23e1ed9869 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.164423] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.164782] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.164782] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.164876] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.164925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.164992] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.165109] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.165186] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.165472] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('0cd25894-1483-4857-b8b7-2f23e1ed9869',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  574.165701] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 40afb8c09d964ec9a68698c650266e2c in queue reply_156db6c4256a4c58884afce15e6f5340
[  574.178321] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 40afb8c09d964ec9a68698c650266e2c
[  574.279289] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.279506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.279635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.279922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.280087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.280355] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.280512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.280630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.280849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.281088] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.281238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.281352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.281560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.281700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.281853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.282152] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.282317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.282440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.282655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.282812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.283055] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.283207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.283324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.283595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.283735] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  574.283976] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 34e6e24a95e04128b2716a2fe0c04af9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  574.289536] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 34e6e24a95e04128b2716a2fe0c04af9
[  574.290573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  574.291049] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  574.291049] env[60890]: ERROR oslo_messaging.rpc.server 
[  574.329034] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-5673dc28-1fe9-4fd8-9b52-8155e5010a70" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.329279] env[60890]: Traceback (most recent call last):
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  574.329309] env[60890]:     timer()
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  574.329309] env[60890]:     cb(*args, **kw)
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  574.329309] env[60890]:     result = function(*args, **kwargs)
[  574.329309] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  574.329309] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  574.329309] env[60890]:     return method(*args, **kwargs)
[  574.329309] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  574.329309] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  574.329309] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  574.329309] env[60890]:     result = self.transport._send(
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  574.329309] env[60890]:     return self._driver.send(target, ctxt, message,
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  574.329309] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  574.329309] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  574.329309] env[60890]:     raise result
[  574.329829] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  574.329829] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  574.521968] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.521968] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.521968] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.521968] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.521968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.522608] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.522608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.522608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.522608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.522907] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  574.523009] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d7d815a987a24f4eb28926f131f50a37 in queue reply_156db6c4256a4c58884afce15e6f5340
[  574.528523] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d7d815a987a24f4eb28926f131f50a37
[  574.608034] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.608034] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.608215] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.608416] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.608513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.608866] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f already deleted.
[  574.608866] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.637647] env[60890]: INFO networking_nsxv3.common.synchronization [None req-32eb25c2-3b35-49f7-a72a-5e829d9c3afa tempest-ServerActionsTestOtherA-363397817 tempest-ServerActionsTestOtherA-363397817-project-member] Enqueued Resource ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f with Priority: HIGHEST for action security_group_rules
[  574.638197] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f with Priority: HIGHEST for action security_group_rules
[  574.638517] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.638645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.638717] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.638954] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.639044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.639201] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.639299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.639362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.639514] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.639595] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('f934cb7a-eebb-4560-bd27-6e115d57ba4f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  574.639778] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7714be21752b420fbf137c9fbc2cc27e in queue reply_156db6c4256a4c58884afce15e6f5340
[  574.664938] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7714be21752b420fbf137c9fbc2cc27e
[  574.667038] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.667170] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.667351] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.667568] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.667658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.667845] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.667960] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.668015] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.668168] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.668246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.668329] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 already deleted.
[  574.668406] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.789213] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.789424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.789553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.789852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.790023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.790296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.790454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.790688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.790792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.791038] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.791225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.791312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.791526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.791678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.791828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.792126] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.792285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.792404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.792614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.792744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  574.793305] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  574.793305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  574.793305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  574.793460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  574.793677] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  574.793895] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e71bd395118c48f98ae6cac9f464bcf6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  574.800020] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e71bd395118c48f98ae6cac9f464bcf6
[  575.025848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  575.026389] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  575.026389] env[60890]: ERROR oslo_messaging.rpc.server 
[  575.162421] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  575.162578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  575.162652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  575.162870] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  575.162968] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.163070] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f already deleted.
[  575.163155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.297018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  575.297525] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  575.297525] env[60890]: ERROR oslo_messaging.rpc.server 
[  575.367129] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  575.367383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  575.367477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  575.367754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  575.367906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.368190] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  575.368351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  575.368475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  575.368697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  575.368938] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  575.369094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  575.369212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  575.369422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  575.369972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.370207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.370477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  575.370639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  575.370772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  575.370985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  575.371115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.371358] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  575.371568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  575.371630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  575.371850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  575.371987] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e70a9ce4-ef18-47d2-8439-b91cf56ba2d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  575.372286] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 103bccdd4026438c965efc793d0584c2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  575.379832] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 103bccdd4026438c965efc793d0584c2
[  575.876194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e70a9ce4-ef18-47d2-8439-b91cf56ba2d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  575.876194] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  575.876194] env[60890]: ERROR oslo_messaging.rpc.server 
[  576.167393] env[60890]: INFO networking_nsxv3.common.synchronization [None req-53a6474e-3f6f-4821-9247-713f6842aaaa None None] Enqueued Resource ID: 61396cf8-5b3d-4655-9ffc-ffc0b40b1ec8 with Priority: HIGHEST for action port
[  576.167710] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 61396cf8-5b3d-4655-9ffc-ffc0b40b1ec8 with Priority: HIGHEST for action port
[  576.168354] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.168487] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.168556] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.168756] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.168845] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.168994] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-61396cf8-5b3d-4655-9ffc-ffc0b40b1ec8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.169090] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-61396cf8-5b3d-4655-9ffc-ffc0b40b1ec8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.169145] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-61396cf8-5b3d-4655-9ffc-ffc0b40b1ec8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.169344] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-61396cf8-5b3d-4655-9ffc-ffc0b40b1ec8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.169424] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('61396cf8-5b3d-4655-9ffc-ffc0b40b1ec8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  576.169608] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 73b65bf814334f9089612781bb213cc0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  576.182092] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 73b65bf814334f9089612781bb213cc0
[  576.304965] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.304965] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.304965] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.304965] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.304965] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.304965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.306924] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  576.307365] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 46d47324411f4b48a671c1cf624e546a in queue reply_156db6c4256a4c58884afce15e6f5340
[  576.312971] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 46d47324411f4b48a671c1cf624e546a
[  576.537991] env[60890]: INFO networking_nsxv3.common.synchronization [None req-b4c0ee72-0d59-4af9-ab0e-c7ce211b33d8 None None] Enqueued Resource ID: 0b754f67-72a2-413f-b926-78d520ce4342 with Priority: HIGHEST for action port
[  576.538485] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0b754f67-72a2-413f-b926-78d520ce4342 with Priority: HIGHEST for action port
[  576.538812] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.538931] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.538995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.539211] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.539286] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.539443] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-0b754f67-72a2-413f-b926-78d520ce4342 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.539549] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-0b754f67-72a2-413f-b926-78d520ce4342" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.539605] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-0b754f67-72a2-413f-b926-78d520ce4342" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.539840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-0b754f67-72a2-413f-b926-78d520ce4342" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.539951] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0b754f67-72a2-413f-b926-78d520ce4342',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  576.540530] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8151c80cadda46a598e6dd8d6ab5f225 in queue reply_156db6c4256a4c58884afce15e6f5340
[  576.552109] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8151c80cadda46a598e6dd8d6ab5f225
[  576.673919] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-61396cf8-5b3d-4655-9ffc-ffc0b40b1ec8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.674160] env[60890]: Traceback (most recent call last):
[  576.674197] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  576.674197] env[60890]:     timer()
[  576.674197] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  576.674197] env[60890]:     cb(*args, **kw)
[  576.674197] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  576.674197] env[60890]:     result = function(*args, **kwargs)
[  576.674197] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  576.674197] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  576.674197] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  576.674197] env[60890]:     return method(*args, **kwargs)
[  576.674197] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  576.674197] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  576.674197] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  576.674197] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  576.674528] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  576.674528] env[60890]:     result = self.transport._send(
[  576.674528] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  576.674528] env[60890]:     return self._driver.send(target, ctxt, message,
[  576.674528] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  576.674528] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  576.674528] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  576.674528] env[60890]:     raise result
[  576.674528] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  576.674528] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  576.698930] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.699139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.699270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.699545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.699696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.699984] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.700193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.700288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.700511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.701152] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.701318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.701440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.701683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.701845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.702036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.702352] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.702519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.702640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.702858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.703028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.703292] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.703449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.703569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.703793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.703938] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  576.704210] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 48d7d76a973e40f4bb9dc58b03279257 in queue reply_156db6c4256a4c58884afce15e6f5340
[  576.714520] env[60890]: INFO networking_nsxv3.common.synchronization [req-36f80583-9b15-43cc-879a-36b63f3d0025 req-1f185f05-f9cc-40d1-8f80-7c09d44a6a01 tempest-ServersWithSpecificFlavorTestJSON-1300140250 tempest-ServersWithSpecificFlavorTestJSON-1300140250-project-member] Enqueued Resource ID: a3a01724-483b-40cb-9132-f26a2a638504 with Priority: HIGHEST for action security_group_members
[  576.715093] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a3a01724-483b-40cb-9132-f26a2a638504 with Priority: HIGHEST for action security_group_members
[  576.715399] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.715518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.715582] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.715769] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.715847] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.716018] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-a3a01724-483b-40cb-9132-f26a2a638504 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.716216] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.716300] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.716507] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.716665] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  576.716767] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  576.716852] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  576.716997] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  576.717073] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.718186] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('a3a01724-483b-40cb-9132-f26a2a638504',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  576.718409] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5a86c734139e493ebe1459f252f9ed1b in queue reply_156db6c4256a4c58884afce15e6f5340
[  576.729300] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 48d7d76a973e40f4bb9dc58b03279257
[  576.746334] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5a86c734139e493ebe1459f252f9ed1b
[  576.810588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  576.811237] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  576.811237] env[60890]: ERROR oslo_messaging.rpc.server 
[  577.043340] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-0b754f67-72a2-413f-b926-78d520ce4342" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.043952] env[60890]: Traceback (most recent call last):
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  577.043952] env[60890]:     timer()
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  577.043952] env[60890]:     cb(*args, **kw)
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  577.043952] env[60890]:     result = function(*args, **kwargs)
[  577.043952] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  577.043952] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  577.043952] env[60890]:     return method(*args, **kwargs)
[  577.043952] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  577.043952] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  577.043952] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  577.043952] env[60890]:     result = self.transport._send(
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  577.043952] env[60890]:     return self._driver.send(target, ctxt, message,
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  577.043952] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  577.043952] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  577.043952] env[60890]:     raise result
[  577.043952] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  577.043952] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  577.126949] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  577.127121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  577.127253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  577.127526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  577.127673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.128072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  577.128185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  577.128217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  577.128492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  577.128745] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  577.128905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  577.129025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  577.129246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  577.129391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.129546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.129805] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  577.130209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  577.130209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  577.130292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  577.130410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.130652] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  577.130804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  577.131146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  577.131180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  577.131300] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  577.131544] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 81a07005e4494ed6bd6a327420efda34 in queue reply_156db6c4256a4c58884afce15e6f5340
[  577.144877] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 81a07005e4494ed6bd6a327420efda34
[  577.206868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  577.207228] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  577.207228] env[60890]: ERROR oslo_messaging.rpc.server 
[  577.220584] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  577.220765] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  577.220808] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  577.221007] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  577.221098] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.221290] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  577.221397] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  577.221462] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  577.221586] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  577.221661] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.221743] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: a3a01724-483b-40cb-9132-f26a2a638504 already deleted.
[  577.221820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.633545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  577.633986] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  577.633986] env[60890]: ERROR oslo_messaging.rpc.server 
[  578.138136] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.138340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.138467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.138736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.138922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.139138] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.139295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.139413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.139632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.139876] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.140093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.140222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.140445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.140581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.140741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.140997] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.141148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.141264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.141473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.141659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.141851] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.142005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.142123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.142328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.142466] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f930ed0-7244-4f64-b29e-b5cb0d82f63d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  578.142728] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 95933e9f3ead442c9a431f392a9ba941 in queue reply_156db6c4256a4c58884afce15e6f5340
[  578.148522] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 95933e9f3ead442c9a431f392a9ba941
[  578.423389] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.423681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.423741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.423995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.424192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.424454] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.424611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.424735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.424972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.425542] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.425781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.425827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.426044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.426214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.426340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.426598] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.426754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.426876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.427246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.427443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.427740] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.427932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.428091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.428456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.428520] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  578.429853] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2ebaf24b18f14c98adfb2a0bad13f777 in queue reply_156db6c4256a4c58884afce15e6f5340
[  578.435226] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2ebaf24b18f14c98adfb2a0bad13f777
[  578.645151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0f930ed0-7244-4f64-b29e-b5cb0d82f63d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  578.645682] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  578.645682] env[60890]: ERROR oslo_messaging.rpc.server 
[  578.931817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  578.932352] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  578.932352] env[60890]: ERROR oslo_messaging.rpc.server 
[  578.964237] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.964447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.964575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.964941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.965097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.965361] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.965525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.965701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.965953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.966206] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.966760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.966953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.967238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.967417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.967610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.968407] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.968407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.968407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.968407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.968610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.968892] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.969098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.969260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.969507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.969687] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  578.969999] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f04a3497d329463088bb1bc3b2efa316 in queue reply_156db6c4256a4c58884afce15e6f5340
[  578.977262] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f04a3497d329463088bb1bc3b2efa316
[  578.994894] env[60890]: INFO networking_nsxv3.common.synchronization [None req-3b1d3788-0149-4336-a1b0-ff3cf62fb8f5 tempest-AttachVolumeShelveTestJSON-2132759461 tempest-AttachVolumeShelveTestJSON-2132759461-project-member] Enqueued Resource ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 with Priority: HIGHEST for action security_group_rules
[  578.995517] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 with Priority: HIGHEST for action security_group_rules
[  578.995874] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.996018] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.996100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.996314] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.996399] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  578.996549] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  578.996646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  578.996707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  578.996920] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  578.997007] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('f80ac04b-9522-404c-9dc2-d2e2b8a228d9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  578.997182] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f63d7806cfe54aef822d8631ebb19d3e in queue reply_156db6c4256a4c58884afce15e6f5340
[  579.010072] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f63d7806cfe54aef822d8631ebb19d3e
[  579.031573] env[60890]: INFO networking_nsxv3.common.synchronization [req-fffd2f23-5125-4a8d-a244-eaf5f62faa95 req-6a606206-4ec6-4b4c-b049-5a3b14240106 tempest-VolumesAssistedSnapshotsTest-1202454958 tempest-VolumesAssistedSnapshotsTest-1202454958-project-member] Enqueued Resource ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 with Priority: HIGHEST for action security_group_members
[  579.032109] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 with Priority: HIGHEST for action security_group_members
[  579.032436] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.032553] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.032702] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.032908] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.032988] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.033141] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-6cb50a73-5b5c-401a-b812-ccab4721d4a6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.033235] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.033294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.033492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.033629] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.033729] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.033785] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.033924] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.034000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.034084] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('6cb50a73-5b5c-401a-b812-ccab4721d4a6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  579.034262] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 56b530bd1f6045cab4feb95b860dfc98 in queue reply_156db6c4256a4c58884afce15e6f5340
[  579.043413] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 56b530bd1f6045cab4feb95b860dfc98
[  579.473495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  579.473495] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  579.473495] env[60890]: ERROR oslo_messaging.rpc.server 
[  579.517979] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.517979] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.517979] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.517979] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.517979] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.517979] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 already deleted.
[  579.517979] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.536492] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.536911] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.537151] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.537466] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.537676] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.537957] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.538176] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.538326] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.538561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.538727] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.538897] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 already deleted.
[  579.539121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.796136] env[60890]: INFO networking_nsxv3.common.synchronization [req-747ba87a-cfaf-471a-a623-86ce852a18e1 req-8aefc12e-eb32-49d2-82d6-fad9300df889 tempest-ServerPasswordTestJSON-354071754 tempest-ServerPasswordTestJSON-354071754-project-member] Enqueued Resource ID: 7eee032f-3cc7-4cbf-9083-687ae6ef6ffb with Priority: HIGHEST for action security_group_members
[  579.796136] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7eee032f-3cc7-4cbf-9083-687ae6ef6ffb with Priority: HIGHEST for action security_group_members
[  579.796136] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.796136] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.796136] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.796136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.796136] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7eee032f-3cc7-4cbf-9083-687ae6ef6ffb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  579.796136] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg dcc3fe356992477aaa2f406972894794 in queue reply_156db6c4256a4c58884afce15e6f5340
[  579.804892] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dcc3fe356992477aaa2f406972894794
[  579.813085] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.813423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.813657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.814032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.814364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.814722] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.815006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.815258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.815578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.815924] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.816315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.816577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.816885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.817135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.817383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.817765] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.818056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.818534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.818847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.819113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.819464] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.819740] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.820648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.822422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.822661] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  579.824220] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e51b094150214968b407939c98a1c384 in queue reply_156db6c4256a4c58884afce15e6f5340
[  579.827652] env[60890]: INFO networking_nsxv3.common.synchronization [None req-abacc4c7-7fce-41b0-b438-382e8317a7c0 None None] Enqueued Resource ID: d49cd012-92e2-4418-8ccf-1c7487cc9390 with Priority: HIGHEST for action port
[  579.828508] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d49cd012-92e2-4418-8ccf-1c7487cc9390 with Priority: HIGHEST for action port
[  579.828907] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.829127] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.829270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.829510] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.829677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  579.829921] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-d49cd012-92e2-4418-8ccf-1c7487cc9390 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  579.830146] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-d49cd012-92e2-4418-8ccf-1c7487cc9390" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  579.830291] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-d49cd012-92e2-4418-8ccf-1c7487cc9390" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  579.830561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-d49cd012-92e2-4418-8ccf-1c7487cc9390" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  579.830725] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d49cd012-92e2-4418-8ccf-1c7487cc9390',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  579.831055] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2bd08bedd5254505add8e2e6fc39d599 in queue reply_156db6c4256a4c58884afce15e6f5340
[  579.834295] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e51b094150214968b407939c98a1c384
[  579.838034] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2bd08bedd5254505add8e2e6fc39d599
[  580.059943] env[60890]: INFO networking_nsxv3.common.synchronization [None req-abfd4376-8019-4170-ac12-f73f0bd84afb tempest-AttachVolumeShelveTestJSON-2132759461 tempest-AttachVolumeShelveTestJSON-2132759461-project-member] Enqueued Resource ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 with Priority: HIGHEST for action security_group_rules
[  580.060480] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 with Priority: HIGHEST for action security_group_rules
[  580.060794] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.060922] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.060984] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.061190] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.061273] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.061418] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.061512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.061568] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.061716] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.061792] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('f80ac04b-9522-404c-9dc2-d2e2b8a228d9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  580.061970] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 54db2af4e2684e2b86d680aee6c770a8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  580.076829] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 54db2af4e2684e2b86d680aee6c770a8
[  580.299184] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.299349] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.299421] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.299644] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.299739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.299938] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.300283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.300283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.300283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.300478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.300478] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7eee032f-3cc7-4cbf-9083-687ae6ef6ffb already deleted.
[  580.300478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7eee032f-3cc7-4cbf-9083-687ae6ef6ffb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.328769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  580.329284] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  580.329284] env[60890]: ERROR oslo_messaging.rpc.server 
[  580.332549] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-d49cd012-92e2-4418-8ccf-1c7487cc9390" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.332653] env[60890]: Traceback (most recent call last):
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  580.332686] env[60890]:     timer()
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  580.332686] env[60890]:     cb(*args, **kw)
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  580.332686] env[60890]:     result = function(*args, **kwargs)
[  580.332686] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  580.332686] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  580.332686] env[60890]:     return method(*args, **kwargs)
[  580.332686] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  580.332686] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  580.332686] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  580.332686] env[60890]:     result = self.transport._send(
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  580.332686] env[60890]:     return self._driver.send(target, ctxt, message,
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  580.332686] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  580.332686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  580.332686] env[60890]:     raise result
[  580.333200] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  580.333200] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  580.582128] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.582296] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.582392] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.582571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.582686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.582783] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 already deleted.
[  580.582865] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.847946] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.848213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.848213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.849468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.849468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.849468] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.849468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.849468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.849468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.849706] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.849833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.850122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.850122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.850263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.850449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.850678] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.850805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.851856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.851856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.851856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.851856] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.851856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.851856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.852145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.852145] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  580.852415] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2b27b95621154a6aaa9be67f2a084715 in queue reply_156db6c4256a4c58884afce15e6f5340
[  580.920334] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.920550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.920923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.920923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.921074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.921375] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.921540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.921677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.922592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.922592] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.922592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.922592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.922991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.922991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.922991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.923253] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.923361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.923482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.923691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.923827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  580.925376] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  580.925376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  580.925376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  580.925376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  580.925376] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  580.925376] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7b78af888ed6426bb49618972cc6c37c in queue reply_156db6c4256a4c58884afce15e6f5340
[  580.965030] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2b27b95621154a6aaa9be67f2a084715
[  580.966999] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7b78af888ed6426bb49618972cc6c37c
[  581.192850] env[60890]: INFO networking_nsxv3.common.synchronization [req-7090f4ef-7a3d-442d-a3ef-d4ac45996f56 req-9fc76e79-aa45-4322-ad2f-26e58b0c9685 tempest-ImagesTestJSON-598093153 tempest-ImagesTestJSON-598093153-project-member] Enqueued Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  581.193434] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  581.193700] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.193821] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.193905] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.194135] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.194209] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.194321] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f0a2229c-87aa-4f25-9157-b1b2e986b23e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.194420] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.194486] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.194665] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.194841] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.197343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.197343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.197343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.197343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.197343] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f0a2229c-87aa-4f25-9157-b1b2e986b23e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  581.197343] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg dd1b1beeb90244889c1cdc5937020e7b in queue reply_156db6c4256a4c58884afce15e6f5340
[  581.217531] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dd1b1beeb90244889c1cdc5937020e7b
[  581.255654] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.255858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.255984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.256295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.256763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.256763] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.256882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.256962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.257177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.257420] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.257576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.257695] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.257904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.258048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.258196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.258445] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.258599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.258719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.258926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.259062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.259297] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.259449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.259568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.259776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.260097] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  581.260173] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c748f928b0364f4abf99d9391134f9ae in queue reply_156db6c4256a4c58884afce15e6f5340
[  581.268836] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c748f928b0364f4abf99d9391134f9ae
[  581.355024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  581.355564] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  581.355564] env[60890]: ERROR oslo_messaging.rpc.server 
[  581.427635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  581.428183] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  581.428183] env[60890]: ERROR oslo_messaging.rpc.server 
[  581.493639] env[60890]: INFO networking_nsxv3.common.synchronization [None req-dff0a343-d673-49be-97cc-c4679d6ec62c None None] Enqueued Resource ID: 40cf3dd8-d58c-4308-8c95-128f257705d8 with Priority: HIGHEST for action port
[  581.493896] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 40cf3dd8-d58c-4308-8c95-128f257705d8 with Priority: HIGHEST for action port
[  581.494455] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.494739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.494739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.494863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.494952] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.495103] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-40cf3dd8-d58c-4308-8c95-128f257705d8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.495195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-40cf3dd8-d58c-4308-8c95-128f257705d8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.495254] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-40cf3dd8-d58c-4308-8c95-128f257705d8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.495466] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-40cf3dd8-d58c-4308-8c95-128f257705d8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.495546] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('40cf3dd8-d58c-4308-8c95-128f257705d8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  581.495749] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 50da1448fab64bc997ef3628de3b3454 in queue reply_156db6c4256a4c58884afce15e6f5340
[  581.503017] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 50da1448fab64bc997ef3628de3b3454
[  581.697136] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.697299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.697370] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.697582] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.697991] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.697991] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  581.698113] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  581.698143] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  581.698294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  581.698397] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.698457] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e already deleted.
[  581.698535] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.761956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  581.762458] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  581.762458] env[60890]: ERROR oslo_messaging.rpc.server 
[  581.997972] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-40cf3dd8-d58c-4308-8c95-128f257705d8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  581.998226] env[60890]: Traceback (most recent call last):
[  581.998257] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  581.998257] env[60890]:     timer()
[  581.998303] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  581.998303] env[60890]:     cb(*args, **kw)
[  581.998303] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  581.998303] env[60890]:     result = function(*args, **kwargs)
[  581.998303] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  581.998303] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  581.998303] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  581.998303] env[60890]:     return method(*args, **kwargs)
[  581.998303] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  581.998303] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  581.998303] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  581.998303] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  581.998303] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  581.998303] env[60890]:     result = self.transport._send(
[  581.998303] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  581.998303] env[60890]:     return self._driver.send(target, ctxt, message,
[  581.998303] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  581.998303] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  581.998303] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  581.998303] env[60890]:     raise result
[  581.998303] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  581.998303] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  582.217108] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.217328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.217616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.217726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.217870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.218135] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.218388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.218446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.218624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.218893] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.219072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.219191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.219402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.219545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.219783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.219952] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.220128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.220246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.220456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.220591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.220835] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.220993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.221114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.221312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.221447] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  582.221679] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg abd6924279074e588ad51d0282c70671 in queue reply_156db6c4256a4c58884afce15e6f5340
[  582.227756] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg abd6924279074e588ad51d0282c70671
[  582.293781] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.294067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.294245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.294595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.294810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.295165] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.295371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.295710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.295833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.296178] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.296381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.296557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.296852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.297073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.297290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.297651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.297872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.298127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.298331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.298524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.298869] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.299090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.299263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.299551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.299753] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  582.300119] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0036575dcf9b4fe39999be76d261db5f in queue reply_156db6c4256a4c58884afce15e6f5340
[  582.306526] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0036575dcf9b4fe39999be76d261db5f
[  582.370188] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.370392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.370520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.370910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.370910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.371174] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.371330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.371448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.371724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.371973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.372148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.372269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.372478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.372623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.372798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.373057] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.373213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.373335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.373570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.373673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.373909] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.374081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.374172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.374415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.374553] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  582.374811] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 713005512dbb4f3fb9b1a73f3b60dbff in queue reply_156db6c4256a4c58884afce15e6f5340
[  582.390895] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 713005512dbb4f3fb9b1a73f3b60dbff
[  582.629321] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.629321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.629321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.629321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.629586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.629695] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.629858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.630006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.630239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.630485] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.630641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.630760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.630981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.631246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.631489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.631760] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.631917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.632141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.632374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.632518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.632769] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  582.633009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  582.633094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  582.633295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  582.633433] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  582.633670] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 160071d7a4914d95965329193ba6389b in queue reply_156db6c4256a4c58884afce15e6f5340
[  582.639360] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 160071d7a4914d95965329193ba6389b
[  582.724132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  582.724754] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  582.724754] env[60890]: ERROR oslo_messaging.rpc.server 
[  582.803567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  582.804139] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  582.804139] env[60890]: ERROR oslo_messaging.rpc.server 
[  582.877187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  582.877689] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  582.877689] env[60890]: ERROR oslo_messaging.rpc.server 
[  583.135686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  583.136316] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  583.136316] env[60890]: ERROR oslo_messaging.rpc.server 
[  583.219630] env[60890]: INFO networking_nsxv3.common.synchronization [req-96919bd4-a1dd-4c5b-913f-4a6bd23ee594 req-002db195-4d45-468e-b53d-3f021a3ce244 tempest-AttachInterfacesUnderV243Test-1652000262 tempest-AttachInterfacesUnderV243Test-1652000262-project-member] Enqueued Resource ID: 5856cafc-e8df-48e6-a28b-0e951067688f with Priority: HIGHEST for action security_group_members
[  583.219630] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5856cafc-e8df-48e6-a28b-0e951067688f with Priority: HIGHEST for action security_group_members
[  583.219630] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.219630] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-5856cafc-e8df-48e6-a28b-0e951067688f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.219630] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.219630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.219630] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('5856cafc-e8df-48e6-a28b-0e951067688f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  583.220286] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9f1a64a85b654b79861e85dacb27213b in queue reply_156db6c4256a4c58884afce15e6f5340
[  583.297890] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9f1a64a85b654b79861e85dacb27213b
[  583.529923] env[60890]: INFO networking_nsxv3.common.synchronization [None req-0d506fb8-059b-4aa6-9f75-cabb7e41b702 None None] Enqueued Resource ID: e6da6bee-b8e3-46a3-8d15-033b66763a3c with Priority: HIGHEST for action port
[  583.530177] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e6da6bee-b8e3-46a3-8d15-033b66763a3c with Priority: HIGHEST for action port
[  583.530764] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.530885] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.530984] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.531144] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.531222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.531398] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-e6da6bee-b8e3-46a3-8d15-033b66763a3c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.531471] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-e6da6bee-b8e3-46a3-8d15-033b66763a3c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.531508] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-e6da6bee-b8e3-46a3-8d15-033b66763a3c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.531703] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-e6da6bee-b8e3-46a3-8d15-033b66763a3c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.531777] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e6da6bee-b8e3-46a3-8d15-033b66763a3c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  583.531949] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3e70287fb1b14d7cb0c1d98f781af6e6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  583.538222] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3e70287fb1b14d7cb0c1d98f781af6e6
[  583.722210] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.722210] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.722210] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.722210] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.722399] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.722472] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.722701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.722789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.722954] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.723035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.723127] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 5856cafc-e8df-48e6-a28b-0e951067688f already deleted.
[  583.723208] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.747179] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.748208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.748208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.748208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.748208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.748208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.748482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.748482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.748642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.748888] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.749171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.749171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.749358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.749503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.749654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.749917] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.750074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.750196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.750406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.750540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.750778] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.750934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.751050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.751424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.751424] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  583.751619] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 549821c3d8434ce4a29905fea0bba756 in queue reply_156db6c4256a4c58884afce15e6f5340
[  583.762294] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 549821c3d8434ce4a29905fea0bba756
[  583.851915] env[60890]: INFO networking_nsxv3.common.synchronization [None req-4aa5fbdb-5c99-401b-ac3a-a56d859acb7b None None] Enqueued Resource ID: b38f6136-8e78-4225-a736-ee2cfe35daae with Priority: HIGHEST for action port
[  583.852216] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b38f6136-8e78-4225-a736-ee2cfe35daae with Priority: HIGHEST for action port
[  583.853014] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.853014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.853014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.853202] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.853286] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  583.853430] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-b38f6136-8e78-4225-a736-ee2cfe35daae taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  583.853521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-b38f6136-8e78-4225-a736-ee2cfe35daae" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  583.853595] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-b38f6136-8e78-4225-a736-ee2cfe35daae" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  583.853780] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-b38f6136-8e78-4225-a736-ee2cfe35daae" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  583.853859] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b38f6136-8e78-4225-a736-ee2cfe35daae',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  583.854034] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 681d63ac22be40e480a6f3837ac4acf0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  583.859830] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 681d63ac22be40e480a6f3837ac4acf0
[  584.034361] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-e6da6bee-b8e3-46a3-8d15-033b66763a3c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.034616] env[60890]: Traceback (most recent call last):
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  584.034714] env[60890]:     timer()
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  584.034714] env[60890]:     cb(*args, **kw)
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  584.034714] env[60890]:     result = function(*args, **kwargs)
[  584.034714] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  584.034714] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  584.034714] env[60890]:     return method(*args, **kwargs)
[  584.034714] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  584.034714] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  584.034714] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  584.034714] env[60890]:     result = self.transport._send(
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  584.034714] env[60890]:     return self._driver.send(target, ctxt, message,
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  584.034714] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  584.034714] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  584.034714] env[60890]:     raise result
[  584.035346] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.035346] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.244033] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.244329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.244329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.244597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.244777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.245057] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.245252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.245332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.245556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.245833] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.246052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.246109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.246330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.246464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.246855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.246924] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.247025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.247146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.247427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.247481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.247714] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.248033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.248033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.248438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.248624] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  584.248892] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 61523c015dfd47baaafb610e20a02300 in queue reply_156db6c4256a4c58884afce15e6f5340
[  584.254249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.254831] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.254831] env[60890]: ERROR oslo_messaging.rpc.server 
[  584.255688] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 61523c015dfd47baaafb610e20a02300
[  584.319524] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.319959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.319959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.320113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.320260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.320512] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.320668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.320791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.321031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.321273] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.321422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.321536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.321737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.321874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.322016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.322291] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.322452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.322592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.322810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.323177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.323177] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.323321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.323435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.323636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.323781] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  584.325037] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7df4277ce5b4458c87ee20d8f440b06e in queue reply_156db6c4256a4c58884afce15e6f5340
[  584.326409] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.326579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.326704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.327067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.327141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.327389] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.327540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.327658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.327968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.328280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.328280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.328409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.328594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.328735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.328879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.332075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.332075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.332075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.332075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.332075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.332075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.332075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.332075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.332075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.332075] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  584.332075] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 22a611c999f448e5b9a00e66fd81a232 in queue reply_156db6c4256a4c58884afce15e6f5340
[  584.332075] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7df4277ce5b4458c87ee20d8f440b06e
[  584.336838] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 22a611c999f448e5b9a00e66fd81a232
[  584.357258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-b38f6136-8e78-4225-a736-ee2cfe35daae" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.357258] env[60890]: Traceback (most recent call last):
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  584.357258] env[60890]:     timer()
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  584.357258] env[60890]:     cb(*args, **kw)
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  584.357258] env[60890]:     result = function(*args, **kwargs)
[  584.357258] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  584.357258] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  584.357258] env[60890]:     return method(*args, **kwargs)
[  584.357258] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  584.357258] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  584.357258] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  584.357258] env[60890]:     result = self.transport._send(
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  584.357258] env[60890]:     return self._driver.send(target, ctxt, message,
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  584.357258] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  584.357258] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  584.357258] env[60890]:     raise result
[  584.357258] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.357258] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.741957] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.742246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.742944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.742944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.742944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.743090] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.743218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.743334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.743611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.744423] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.745322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.745741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.745811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.745929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.746085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.746342] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.746494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.746613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.746865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.747032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.747284] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.747782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.748045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.748366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.748571] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  584.748885] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d50786fa0e6d4fdfb073deacca1a5b6c in queue reply_156db6c4256a4c58884afce15e6f5340
[  584.751324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.752233] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.752233] env[60890]: ERROR oslo_messaging.rpc.server 
[  584.764535] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d50786fa0e6d4fdfb073deacca1a5b6c
[  584.831947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.832472] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.832472] env[60890]: ERROR oslo_messaging.rpc.server 
[  584.833851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.834051] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  584.834051] env[60890]: ERROR oslo_messaging.rpc.server 
[  584.844447] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.845231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.845231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.845231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.845231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.846127] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.846127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.846127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.846127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.846308] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.846382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.846502] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.846714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.846859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.847015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.847283] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.847497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.847546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.847804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.847878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.848130] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.848283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.848400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.848602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.848739] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  584.849022] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c9c4c16d462948f89565b1537060c977 in queue reply_156db6c4256a4c58884afce15e6f5340
[  584.858744] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c9c4c16d462948f89565b1537060c977
[  584.905994] env[60890]: INFO networking_nsxv3.common.synchronization [None req-466e8562-7e3f-45ab-bc5a-632bb691515a None None] Enqueued Resource ID: 9821a78b-7541-4f2f-9763-b21a079108d2 with Priority: HIGHEST for action port
[  584.906525] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9821a78b-7541-4f2f-9763-b21a079108d2 with Priority: HIGHEST for action port
[  584.906842] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.906958] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.907022] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.907208] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.907287] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  584.907428] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-9821a78b-7541-4f2f-9763-b21a079108d2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  584.907516] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-9821a78b-7541-4f2f-9763-b21a079108d2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  584.908286] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-9821a78b-7541-4f2f-9763-b21a079108d2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  584.908286] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-9821a78b-7541-4f2f-9763-b21a079108d2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  584.908286] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9821a78b-7541-4f2f-9763-b21a079108d2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  584.908286] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 16112a9ad35e4012a717f57dd28c83a2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  584.922084] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 16112a9ad35e4012a717f57dd28c83a2
[  585.250881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  585.251359] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  585.251359] env[60890]: ERROR oslo_messaging.rpc.server 
[  585.262341] env[60890]: INFO networking_nsxv3.common.synchronization [req-a0e093bf-a434-4311-8ef8-7b23964970a1 req-d61a227a-7277-4003-80f7-cd2c8f70f68e tempest-ServerRescueNegativeTestJSON-1640230806 tempest-ServerRescueNegativeTestJSON-1640230806-project-member] Enqueued Resource ID: e314c230-88be-4118-8948-b5f480080893 with Priority: HIGHEST for action security_group_members
[  585.262849] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e314c230-88be-4118-8948-b5f480080893 with Priority: HIGHEST for action security_group_members
[  585.263158] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.263282] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.263351] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.263551] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.263637] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.263954] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e314c230-88be-4118-8948-b5f480080893 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.263954] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.263954] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.264163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.264274] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.264367] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.264427] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.264567] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.264647] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.264766] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e314c230-88be-4118-8948-b5f480080893',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  585.264952] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6810e05bb99a4546939e0782bf39bdd7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  585.281253] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6810e05bb99a4546939e0782bf39bdd7
[  585.351535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  585.351941] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  585.351941] env[60890]: ERROR oslo_messaging.rpc.server 
[  585.391215] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.391890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.391890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.391890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.392209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.392379] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.392379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.392501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.392823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.392989] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.393134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.393257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.393463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.393813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.393813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.393993] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.394150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.394268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.394474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.394608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.395490] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.395490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.395490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.395490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.395490] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  585.395726] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 882a2d2d3cb148758135d77e5d2829dc in queue reply_156db6c4256a4c58884afce15e6f5340
[  585.402003] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 882a2d2d3cb148758135d77e5d2829dc
[  585.413128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-9821a78b-7541-4f2f-9763-b21a079108d2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.413350] env[60890]: Traceback (most recent call last):
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  585.413394] env[60890]:     timer()
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  585.413394] env[60890]:     cb(*args, **kw)
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  585.413394] env[60890]:     result = function(*args, **kwargs)
[  585.413394] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  585.413394] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  585.413394] env[60890]:     return method(*args, **kwargs)
[  585.413394] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  585.413394] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  585.413394] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  585.413394] env[60890]:     result = self.transport._send(
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  585.413394] env[60890]:     return self._driver.send(target, ctxt, message,
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  585.413394] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  585.413394] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  585.413394] env[60890]:     raise result
[  585.414170] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  585.414170] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  585.554833] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.555037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.555160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.555423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.555557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.555809] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.555954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.556087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.556298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.556565] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.556713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.556793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.556995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.557146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.557294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.557534] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.557690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.557807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.558068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.558220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.558461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f81b71b7-fe2a-4b40-aad8-dbd948f09710 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.558755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.558755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.560766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.560766] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f81b71b7-fe2a-4b40-aad8-dbd948f09710',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  585.560766] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 389d4c7590074eda8c99b46955fb212f in queue reply_156db6c4256a4c58884afce15e6f5340
[  585.570439] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 389d4c7590074eda8c99b46955fb212f
[  585.636306] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.636507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.636635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.636907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.637075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.637339] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.637496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.637612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.637860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.638078] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.638227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.638341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.638545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.638687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.638834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.639082] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.639229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.639343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.639541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.639668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.639903] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.640081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.640200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.640402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.640534] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  585.640777] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1aeee204385c42a585e7efd055812736 in queue reply_156db6c4256a4c58884afce15e6f5340
[  585.692298] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1aeee204385c42a585e7efd055812736
[  585.772421] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.772582] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.772658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.772894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.773386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.773386] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  585.773386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  585.773386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  585.773582] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  585.773582] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.773642] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e314c230-88be-4118-8948-b5f480080893 already deleted.
[  585.773705] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.898263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  585.898485] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  585.898485] env[60890]: ERROR oslo_messaging.rpc.server 
[  586.003656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.003899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.004079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.004349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.004495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.004782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.005352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.005352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.005352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.005509] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.005671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.005811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.006034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.006179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.006329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.006580] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.006731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.006844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.007049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.007179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.007417] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.007568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.007702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.007886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.009076] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  586.009257] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c6025cc6c3dc4cd590fca08a5e3ea7f9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  586.012295] env[60890]: INFO networking_nsxv3.common.synchronization [None req-0b8a7b61-403d-4c81-8b5e-8e172e7a4bb8 None None] Enqueued Resource ID: aa4fe69e-8c82-4a7d-9439-5bf23d233cee with Priority: HIGHEST for action port
[  586.013033] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: aa4fe69e-8c82-4a7d-9439-5bf23d233cee with Priority: HIGHEST for action port
[  586.014612] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.014612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.014612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.015151] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.015151] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.015151] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-aa4fe69e-8c82-4a7d-9439-5bf23d233cee taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.015312] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-aa4fe69e-8c82-4a7d-9439-5bf23d233cee" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.015312] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-aa4fe69e-8c82-4a7d-9439-5bf23d233cee" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.015786] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-aa4fe69e-8c82-4a7d-9439-5bf23d233cee" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.015786] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('aa4fe69e-8c82-4a7d-9439-5bf23d233cee',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  586.015786] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg eb4e29a1e44e426db4a86e58e610011e in queue reply_156db6c4256a4c58884afce15e6f5340
[  586.016611] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c6025cc6c3dc4cd590fca08a5e3ea7f9
[  586.024923] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eb4e29a1e44e426db4a86e58e610011e
[  586.062621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f81b71b7-fe2a-4b40-aad8-dbd948f09710" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.063391] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.063391] env[60890]: ERROR oslo_messaging.rpc.server 
[  586.144159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.144159] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.144159] env[60890]: ERROR oslo_messaging.rpc.server 
[  586.188679] env[60890]: INFO networking_nsxv3.common.synchronization [None req-e506a118-feb4-476d-84c1-145fa44359dd None None] Enqueued Resource ID: 3adfc372-0ba1-49a9-ab2a-3b65e27ee7e6 with Priority: HIGHEST for action port
[  586.188977] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 3adfc372-0ba1-49a9-ab2a-3b65e27ee7e6 with Priority: HIGHEST for action port
[  586.189561] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.189692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.189776] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.189957] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.190041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.190186] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-3adfc372-0ba1-49a9-ab2a-3b65e27ee7e6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.190282] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-3adfc372-0ba1-49a9-ab2a-3b65e27ee7e6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.190343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-3adfc372-0ba1-49a9-ab2a-3b65e27ee7e6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.190552] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-3adfc372-0ba1-49a9-ab2a-3b65e27ee7e6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.190636] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3adfc372-0ba1-49a9-ab2a-3b65e27ee7e6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  586.190827] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8f65a3fe094a4d8081561b5597e7b37f in queue reply_156db6c4256a4c58884afce15e6f5340
[  586.299115] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8f65a3fe094a4d8081561b5597e7b37f
[  586.306567] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.306897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.307096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.307421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.307636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.308050] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.308247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.309503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.309503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.309503] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.309503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.309503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.310115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.310341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.310556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.310867] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.311084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.311254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.311541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.312260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.312630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.312868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.313044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.313339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.315219] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  586.316415] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 101a987e74b7431b9634864119b98844 in queue reply_156db6c4256a4c58884afce15e6f5340
[  586.331578] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 101a987e74b7431b9634864119b98844
[  586.352687] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.352687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.352838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.353168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.353168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.353390] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.353544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.353662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.353884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.354114] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.354274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.354381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.354583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.354796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.354958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.355208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.355362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.355481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.355705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.355843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.356089] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.356244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.356361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.356575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.356706] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  586.356934] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 98cf8a0a469948e99451ed2446c219ee in queue reply_156db6c4256a4c58884afce15e6f5340
[  586.367005] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 98cf8a0a469948e99451ed2446c219ee
[  586.429152] env[60890]: INFO networking_nsxv3.common.synchronization [req-3fff35ff-618e-48a3-bb5c-136d8cf1b5e6 req-f9664936-054d-40a0-9eaa-9b52da906362 tempest-ServersAdminTestJSON-603115879 tempest-ServersAdminTestJSON-603115879-project-member] Enqueued Resource ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd with Priority: HIGHEST for action security_group_members
[  586.429152] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd with Priority: HIGHEST for action security_group_members
[  586.429152] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.429152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.429152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.429152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.429152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.429152] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-abd91d97-a4e5-45fe-9581-3f1337cd29cd taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.429152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.429152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.429152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.429152] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.429557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.429557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.429557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.429557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.429557] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('abd91d97-a4e5-45fe-9581-3f1337cd29cd',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  586.429827] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9e48b174ca664ca0888c5fcf189846c7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  586.441189] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9e48b174ca664ca0888c5fcf189846c7
[  586.511423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.512029] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.512029] env[60890]: ERROR oslo_messaging.rpc.server 
[  586.518154] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-aa4fe69e-8c82-4a7d-9439-5bf23d233cee" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.518351] env[60890]: Traceback (most recent call last):
[  586.518383] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  586.518383] env[60890]:     timer()
[  586.518383] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  586.518383] env[60890]:     cb(*args, **kw)
[  586.518383] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  586.518383] env[60890]:     result = function(*args, **kwargs)
[  586.518383] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  586.518383] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  586.518383] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  586.518383] env[60890]:     return method(*args, **kwargs)
[  586.518618] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  586.518618] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  586.518618] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  586.518618] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  586.518618] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  586.518618] env[60890]:     result = self.transport._send(
[  586.518618] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  586.518618] env[60890]:     return self._driver.send(target, ctxt, message,
[  586.518618] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  586.518618] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  586.518618] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  586.518618] env[60890]:     raise result
[  586.518618] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.518618] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.692983] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-3adfc372-0ba1-49a9-ab2a-3b65e27ee7e6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.693216] env[60890]: Traceback (most recent call last):
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  586.693261] env[60890]:     timer()
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  586.693261] env[60890]:     cb(*args, **kw)
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  586.693261] env[60890]:     result = function(*args, **kwargs)
[  586.693261] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  586.693261] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  586.693261] env[60890]:     return method(*args, **kwargs)
[  586.693261] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  586.693261] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  586.693261] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  586.693261] env[60890]:     result = self.transport._send(
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  586.693261] env[60890]:     return self._driver.send(target, ctxt, message,
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  586.693261] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  586.693261] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  586.693261] env[60890]:     raise result
[  586.693922] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.693922] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.821448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.821955] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.821955] env[60890]: ERROR oslo_messaging.rpc.server 
[  586.857903] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.858118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.858249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.858516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.858663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.858921] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.859072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.859191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.859701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.859701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.859827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.859897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.860131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.860276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.860469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.860669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.860822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.860942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.861149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.861284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.861519] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.861670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.861785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.862013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.862156] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  586.862385] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6df2f7ff43c74fa9b1bad3615a240fe9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  586.864297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.864755] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  586.864755] env[60890]: ERROR oslo_messaging.rpc.server 
[  586.867723] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6df2f7ff43c74fa9b1bad3615a240fe9
[  586.932127] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.933087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.933087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.933087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.933087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.933087] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  586.933087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  586.933087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  586.933395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  586.933395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  586.933395] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd already deleted.
[  586.933395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.301596] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.301870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.301911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.302177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.302318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.302590] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.302776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.302900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.303113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.303350] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.303539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.303604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.303805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.303938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.304093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.304372] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.304532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.304645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.305029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.305177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.305430] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.305720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.305720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.305926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.306061] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  587.306289] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 57d4fc89c6a34ad69a5b6955f7ead030 in queue reply_156db6c4256a4c58884afce15e6f5340
[  587.313200] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 57d4fc89c6a34ad69a5b6955f7ead030
[  587.367166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  587.367743] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  587.367743] env[60890]: ERROR oslo_messaging.rpc.server 
[  587.661855] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.662055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.662194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.662471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.662617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.662901] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.663059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.663177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.663451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.663714] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.663873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.664030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.664234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.664381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.664532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.664808] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.664965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.665082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.665313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.665451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.666005] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  587.666166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  587.666285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  587.666534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  587.666675] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  587.666911] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6ad7090928f24b73bb764c8579539a42 in queue reply_156db6c4256a4c58884afce15e6f5340
[  587.672622] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6ad7090928f24b73bb764c8579539a42
[  587.809241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  587.809793] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  587.809793] env[60890]: ERROR oslo_messaging.rpc.server 
[  588.006957] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.007133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.007260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.007527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.007675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.007927] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.008101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.008306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.008876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.008876] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.008876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.008979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.009195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.009519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.009519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.009791] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.010011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.010011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.010263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.010331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.010602] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.011178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.011322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.011658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.011826] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.012133] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7f1c57f6ffad42d2a5997afe72ceb6a1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.017609] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7f1c57f6ffad42d2a5997afe72ceb6a1
[  588.054828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.055096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.055260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.055559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.055768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.056104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.056305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.056733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.056733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.058226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.066368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.066368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.066368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.066368] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.066368] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 655e1617259a4a9ba1c341c72270ecb8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.074990] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 655e1617259a4a9ba1c341c72270ecb8
[  588.169514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.170046] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.170046] env[60890]: ERROR oslo_messaging.rpc.server 
[  588.195300] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.195504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.195636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.195912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.196212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.196499] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.196662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.196798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.197117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.197375] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.197721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.197721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.197880] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.198083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.198172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.198424] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.198583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.198705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.198924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.199061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.199435] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.199611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.199736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.200000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.200156] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.200393] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg db9051bd4e2a4258aa843ca68bc261a2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.206135] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg db9051bd4e2a4258aa843ca68bc261a2
[  588.213224] env[60890]: INFO networking_nsxv3.common.synchronization [None req-4f18230a-d3a9-4e43-a8f5-fae6e34ec3a4 None None] Enqueued Resource ID: 935cd9c0-94e7-40a5-b7a2-be872b88fc1e with Priority: HIGHEST for action port
[  588.213578] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 935cd9c0-94e7-40a5-b7a2-be872b88fc1e with Priority: HIGHEST for action port
[  588.213893] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.214013] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.214080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.214272] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.214356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.214503] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-935cd9c0-94e7-40a5-b7a2-be872b88fc1e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.214594] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-935cd9c0-94e7-40a5-b7a2-be872b88fc1e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.214662] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-935cd9c0-94e7-40a5-b7a2-be872b88fc1e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.214914] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-935cd9c0-94e7-40a5-b7a2-be872b88fc1e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.214969] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('935cd9c0-94e7-40a5-b7a2-be872b88fc1e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.215132] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 85d6686aab8843d5a132ba33c64ee067 in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.235352] env[60890]: INFO networking_nsxv3.common.synchronization [req-72ba3ef2-cd37-4d4e-965e-fd5ded9878c1 req-113c6747-f6d6-434e-a955-bdc16414969e tempest-ListImageFiltersTestJSON-1201112359 tempest-ListImageFiltersTestJSON-1201112359-project-member] Enqueued Resource ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c with Priority: HIGHEST for action security_group_members
[  588.235352] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c with Priority: HIGHEST for action security_group_members
[  588.235352] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.235352] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e0b6278a-c9cf-4011-aaba-b3bfed71204c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.235352] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.235352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.236060] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.236060] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.236060] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e0b6278a-c9cf-4011-aaba-b3bfed71204c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.236060] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3a116fd6db064981b155d861e0d37b54 in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.246092] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 85d6686aab8843d5a132ba33c64ee067
[  588.255313] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3a116fd6db064981b155d861e0d37b54
[  588.278346] env[60890]: INFO networking_nsxv3.common.synchronization [req-954f4716-cea8-4ea6-a0b5-c481f4be5ad0 req-88bc03d3-ff8b-43ac-9dc4-2263cffad43e tempest-ServersAdminTestJSON-603115879 tempest-ServersAdminTestJSON-603115879-project-member] Enqueued Resource ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd with Priority: HIGHEST for action security_group_members
[  588.278346] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd with Priority: HIGHEST for action security_group_members
[  588.278346] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.278346] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-abd91d97-a4e5-45fe-9581-3f1337cd29cd taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.278346] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.278346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.278843] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.278843] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.278843] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('abd91d97-a4e5-45fe-9581-3f1337cd29cd',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.278843] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 59d347a9de9e445c8651b04dd5e36c0c in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.287747] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 59d347a9de9e445c8651b04dd5e36c0c
[  588.301259] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.301435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.301564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.301845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.302096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.302244] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.302393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.302511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.302746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.302995] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.303225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.303359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.303569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.303720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.303870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.304137] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.304286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.304401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.304605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.304756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.305139] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.305313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.305410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.305616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.305749] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.305977] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5bbbf55591e0414881b557ec18c711d0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.311612] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.311812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.311875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.312114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.312258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.312491] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.312648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.313281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.313515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.314021] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.314191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.314317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.314529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.314714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.314855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.315156] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.315259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.315376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.315583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.315803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.315982] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.316151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.316271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.317089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.317089] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.317089] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 598589c0fa4f462cae734398689c54d8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.320857] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5bbbf55591e0414881b557ec18c711d0
[  588.327511] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 598589c0fa4f462cae734398689c54d8
[  588.517465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.518187] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.518187] env[60890]: ERROR oslo_messaging.rpc.server 
[  588.570415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.571069] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.571069] env[60890]: ERROR oslo_messaging.rpc.server 
[  588.660028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.660028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.660028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.660028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.660555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.660555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.660555] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.660652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.660742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.660915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.661049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.661323] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.661491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.661611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.661863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.661957] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  588.662179] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 06229009e7be42609a1f9003d2235a92 in queue reply_156db6c4256a4c58884afce15e6f5340
[  588.669087] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 06229009e7be42609a1f9003d2235a92
[  588.702794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.703361] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.703361] env[60890]: ERROR oslo_messaging.rpc.server 
[  588.717849] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-935cd9c0-94e7-40a5-b7a2-be872b88fc1e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.718075] env[60890]: Traceback (most recent call last):
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  588.718142] env[60890]:     timer()
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  588.718142] env[60890]:     cb(*args, **kw)
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  588.718142] env[60890]:     result = function(*args, **kwargs)
[  588.718142] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  588.718142] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  588.718142] env[60890]:     return method(*args, **kwargs)
[  588.718142] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  588.718142] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  588.718142] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  588.718142] env[60890]:     result = self.transport._send(
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  588.718142] env[60890]:     return self._driver.send(target, ctxt, message,
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  588.718142] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  588.718142] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  588.718142] env[60890]:     raise result
[  588.718142] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.718142] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.746356] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.746356] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.746356] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e0b6278a-c9cf-4011-aaba-b3bfed71204c already deleted.
[  588.746356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e0b6278a-c9cf-4011-aaba-b3bfed71204c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.781322] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.781322] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.781322] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.781322] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.781505] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.781543] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  588.782401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  588.782401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  588.782401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  588.782401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.782401] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd already deleted.
[  588.782401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.819689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.820233] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.820233] env[60890]: ERROR oslo_messaging.rpc.server 
[  588.821462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.821881] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  588.821881] env[60890]: ERROR oslo_messaging.rpc.server 
[  589.164641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  589.165306] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  589.165306] env[60890]: ERROR oslo_messaging.rpc.server 
[  589.257836] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.258045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.258176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.258446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.258594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.258851] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.259011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.259133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.259352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.259599] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.259757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.259878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.260103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.260252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.260405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.260659] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.260825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.260969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.261181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.261317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.261568] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.261755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.261906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.262216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.262293] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  589.262483] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 19e3a1956b9143edb0563bf99a6f15e2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  589.268058] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 19e3a1956b9143edb0563bf99a6f15e2
[  589.302225] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.302485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.302613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.302894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.303051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.303311] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.303468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.303587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.303804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.304070] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.304232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.304351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.304557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.304767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.304933] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.305187] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.305344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.305463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.305685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.305840] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.306078] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.306233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.306352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.306558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.306694] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  589.306925] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0815265394e8403a986d1154fa6d17d6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  589.311587] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0815265394e8403a986d1154fa6d17d6
[  589.401054] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.401154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.401270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.401483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.401631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.401887] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.402045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.402164] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.402377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.402621] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.402778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.402897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.403105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.403249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.403400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.403647] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.403806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.403923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.404179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.404320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.404559] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.405199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.405330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.405544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.405702] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  589.405948] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0b7f90bae998410eaede20ac1f5f22c6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  589.410948] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0b7f90bae998410eaede20ac1f5f22c6
[  589.525549] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.525849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.525988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.526262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.526411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.526678] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.526836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.526957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.527175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.527418] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.527571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.527688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.527896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.528055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.528207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.528457] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.528611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.528731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.528941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.529076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.529314] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.529465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.529584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.529789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.529930] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  589.530166] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a70244ec026f405fa0a77db78a6d631d in queue reply_156db6c4256a4c58884afce15e6f5340
[  589.555280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.555531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.555747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.556136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.556312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.556578] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.556735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.556853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.557085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.557331] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.557486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.557622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.557850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.557994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.558143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.558391] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.558543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.558665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.558872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.559003] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.559242] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.559390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.559505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.559863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.560021] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  589.560263] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 11c29f8e4688496d96b7c0b6e1a48a25 in queue reply_156db6c4256a4c58884afce15e6f5340
[  589.564588] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a70244ec026f405fa0a77db78a6d631d
[  589.568618] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 11c29f8e4688496d96b7c0b6e1a48a25
[  589.765334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  589.765959] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  589.765959] env[60890]: ERROR oslo_messaging.rpc.server 
[  589.809284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  589.809820] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  589.809820] env[60890]: ERROR oslo_messaging.rpc.server 
[  589.907790] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.907997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.908147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.908424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.908573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.908836] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.908996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.909118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.909341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.909589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.909750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.909870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.910081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.910227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.910379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.910628] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.910815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.910896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.911175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.911263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.911502] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.911645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.911787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.911995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.912228] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  589.912345] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3caea7818a2d495190fb2c13069b2168 in queue reply_156db6c4256a4c58884afce15e6f5340
[  589.917092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  589.917092] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  589.917092] env[60890]: ERROR oslo_messaging.rpc.server 
[  589.920918] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3caea7818a2d495190fb2c13069b2168
[  589.974147] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.974826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.974826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.974826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.974937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.975181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.975411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.975545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.975876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.976051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.976218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.976333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.976540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.976719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.985420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.985420] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.985420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.985420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.985420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.985420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  589.985420] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  589.985420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  589.985420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  589.985420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  589.985420] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  589.985420] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 40604f56bf0b4ccebc4d4cf46ddee5fc in queue reply_156db6c4256a4c58884afce15e6f5340
[  589.993098] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 40604f56bf0b4ccebc4d4cf46ddee5fc
[  590.032445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.032984] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.032984] env[60890]: ERROR oslo_messaging.rpc.server 
[  590.062987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.063490] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.063490] env[60890]: ERROR oslo_messaging.rpc.server 
[  590.194126] env[60890]: INFO networking_nsxv3.common.synchronization [None req-1a84a53b-406e-4849-91ee-ec0e728d4a6f None None] Enqueued Resource ID: 4355fe65-d621-48ec-ba3b-17ba17881635 with Priority: HIGHEST for action port
[  590.194398] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 4355fe65-d621-48ec-ba3b-17ba17881635 with Priority: HIGHEST for action port
[  590.195011] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.195137] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.195208] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.195409] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.195494] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.195642] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-4355fe65-d621-48ec-ba3b-17ba17881635 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.195740] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-4355fe65-d621-48ec-ba3b-17ba17881635" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.195803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-4355fe65-d621-48ec-ba3b-17ba17881635" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.196513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-4355fe65-d621-48ec-ba3b-17ba17881635" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.196513] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4355fe65-d621-48ec-ba3b-17ba17881635',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  590.196513] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f7b89395688041f3bfd2bb5515073ff3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  590.221824] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f7b89395688041f3bfd2bb5515073ff3
[  590.228674] env[60890]: INFO networking_nsxv3.common.synchronization [None req-5a015489-4558-4fb9-8a4e-2776c64f86a9 None None] Enqueued Resource ID: 9e42452d-0d7e-4400-84ba-f93325389ca5 with Priority: HIGHEST for action port
[  590.229509] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9e42452d-0d7e-4400-84ba-f93325389ca5 with Priority: HIGHEST for action port
[  590.229650] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.229780] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.229851] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.230044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.230134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.230460] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-9e42452d-0d7e-4400-84ba-f93325389ca5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.230460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-9e42452d-0d7e-4400-84ba-f93325389ca5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.230460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-9e42452d-0d7e-4400-84ba-f93325389ca5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.230634] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-9e42452d-0d7e-4400-84ba-f93325389ca5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.230715] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9e42452d-0d7e-4400-84ba-f93325389ca5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  590.230895] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 34b70706127c458891163a0ff2e3cbfb in queue reply_156db6c4256a4c58884afce15e6f5340
[  590.315939] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 34b70706127c458891163a0ff2e3cbfb
[  590.333640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.333640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.333640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.333640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.333640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.333640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.334645] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  590.335056] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 745cae42e81248f6a3daad9f4556d792 in queue reply_156db6c4256a4c58884afce15e6f5340
[  590.341630] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 745cae42e81248f6a3daad9f4556d792
[  590.368046] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.368917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.369157] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.369503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.369981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.370377] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.370662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.370885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.371201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.371560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.372462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.372787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.373115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.373582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.374039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.374415] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.374825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.375060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.375381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.375623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.375989] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.376301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.376525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.376826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.377067] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  590.377493] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e479cb19330a4a709725bfd97912a1e2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  590.382783] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e479cb19330a4a709725bfd97912a1e2
[  590.383532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.384274] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.384274] env[60890]: ERROR oslo_messaging.rpc.server 
[  590.418981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.419713] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.419713] env[60890]: ERROR oslo_messaging.rpc.server 
[  590.483369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.483890] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.483890] env[60890]: ERROR oslo_messaging.rpc.server 
[  590.698508] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-4355fe65-d621-48ec-ba3b-17ba17881635" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.698750] env[60890]: Traceback (most recent call last):
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  590.698815] env[60890]:     timer()
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  590.698815] env[60890]:     cb(*args, **kw)
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  590.698815] env[60890]:     result = function(*args, **kwargs)
[  590.698815] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  590.698815] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.698815] env[60890]:     return method(*args, **kwargs)
[  590.698815] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.698815] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.698815] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.698815] env[60890]:     result = self.transport._send(
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.698815] env[60890]:     return self._driver.send(target, ctxt, message,
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.698815] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.698815] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.698815] env[60890]:     raise result
[  590.698815] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.698815] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.736480] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-9e42452d-0d7e-4400-84ba-f93325389ca5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.736658] env[60890]: Traceback (most recent call last):
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  590.736715] env[60890]:     timer()
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  590.736715] env[60890]:     cb(*args, **kw)
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  590.736715] env[60890]:     result = function(*args, **kwargs)
[  590.736715] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  590.736715] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.736715] env[60890]:     return method(*args, **kwargs)
[  590.736715] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.736715] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.736715] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.736715] env[60890]:     result = self.transport._send(
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.736715] env[60890]:     return self._driver.send(target, ctxt, message,
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.736715] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.736715] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.736715] env[60890]:     raise result
[  590.737263] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.737263] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.768979] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.768979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.768979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.768979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.768979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.769309] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.769309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.769423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.769630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.769883] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.770326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.770326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.770549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.770549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.770621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.770882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.771027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.771139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.771339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.771467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.771697] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.776038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.776172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.776407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.776548] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  590.776788] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg dbacc43c07b9486aa7c28cc48912e53f in queue reply_156db6c4256a4c58884afce15e6f5340
[  590.787259] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.787259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.787427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.787575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.787719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.787982] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.788199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.788325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.788538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.788813] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.788971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.789089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.789299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.789440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.789585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.789830] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.789981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.790104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.790308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.790438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.790679] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.790831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.790946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.791535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.791746] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  590.791967] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 029f928363b141f7b65a69d788cfd902 in queue reply_156db6c4256a4c58884afce15e6f5340
[  590.793336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.793498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.793634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.793853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.793993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.794233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-446 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.794387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.797525] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-446" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.797525] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.797525] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3aa8304b-d936-47b0-9754-87eeab985f26 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  590.797525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  590.798151] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3aa8304b-d936-47b0-9754-87eeab985f26',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  590.798151] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 20088b6cc67b424693e444949ca302d3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  590.799165] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dbacc43c07b9486aa7c28cc48912e53f
[  590.799711] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 029f928363b141f7b65a69d788cfd902
[  590.800319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.800761] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.800761] env[60890]: ERROR oslo_messaging.rpc.server 
[  590.813071] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 20088b6cc67b424693e444949ca302d3
[  590.838141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.838316] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  590.838316] env[60890]: ERROR oslo_messaging.rpc.server 
[  591.103374] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.103603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.103732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.104009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.104172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.104731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.104731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.104853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.105035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.105285] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.105440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.105559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.105770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.105916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.106122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.106436] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.106598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.106719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.106937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.107061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.107301] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.107454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.107573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.107776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.107913] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  591.109203] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fe565c18f8f74e458b76876ba7576aad in queue reply_156db6c4256a4c58884afce15e6f5340
[  591.114483] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fe565c18f8f74e458b76876ba7576aad
[  591.251902] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.252147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.252281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.252554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.252706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.252970] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.253131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.253321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.253471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.253718] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.253877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.253996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.254212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.254384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.254542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.254827] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.254991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.255113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.255326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.255464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.255707] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.255867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.255988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.256218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.256354] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  591.256581] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1fb0650588524e7cbbb79efc061360d2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  591.261678] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1fb0650588524e7cbbb79efc061360d2
[  591.284508] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  591.285073] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  591.285073] env[60890]: ERROR oslo_messaging.rpc.server 
[  591.303455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3aa8304b-d936-47b0-9754-87eeab985f26" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  591.303974] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  591.303974] env[60890]: ERROR oslo_messaging.rpc.server 
[  591.595467] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.595791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.595839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.598160] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.598160] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.598160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.598807] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.598807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.598807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.598807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.598949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.599188] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.599347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.599465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.599671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.599813] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  591.600061] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f249e370581048f88ec89c0d341ea07b in queue reply_156db6c4256a4c58884afce15e6f5340
[  591.606127] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f249e370581048f88ec89c0d341ea07b
[  591.610427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  591.610886] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  591.610886] env[60890]: ERROR oslo_messaging.rpc.server 
[  591.758810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  591.758904] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  591.758904] env[60890]: ERROR oslo_messaging.rpc.server 
[  591.916233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.916233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.916233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.916233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.916233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.916233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.916233] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  591.916233] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 72430e453e0f43edbb21ac73fe23bb14 in queue reply_156db6c4256a4c58884afce15e6f5340
[  591.923615] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 72430e453e0f43edbb21ac73fe23bb14
[  591.978535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.978535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.978535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.978535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  591.978535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  591.978535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  591.978535] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  591.978535] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c7c79f6ec0c145efa900bb32b6c871a8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  591.984102] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c7c79f6ec0c145efa900bb32b6c871a8
[  592.074095] env[60890]: INFO networking_nsxv3.common.synchronization [None req-9843214e-d7b8-4e23-84c2-fa0de1610dca None None] Enqueued Resource ID: d6ce390a-e3ed-4ffb-af33-6be77969f1b2 with Priority: HIGHEST for action port
[  592.074278] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d6ce390a-e3ed-4ffb-af33-6be77969f1b2 with Priority: HIGHEST for action port
[  592.074874] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.075003] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.075083] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.075263] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.075346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.075491] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-d6ce390a-e3ed-4ffb-af33-6be77969f1b2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.076352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-d6ce390a-e3ed-4ffb-af33-6be77969f1b2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.076352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-d6ce390a-e3ed-4ffb-af33-6be77969f1b2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.076352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-d6ce390a-e3ed-4ffb-af33-6be77969f1b2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.076352] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d6ce390a-e3ed-4ffb-af33-6be77969f1b2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  592.076352] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 802a57c48fdf456ead3e6cc93b5a13e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  592.082321] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 802a57c48fdf456ead3e6cc93b5a13e0
[  592.102459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.102459] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.102459] env[60890]: ERROR oslo_messaging.rpc.server 
[  592.138983] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.139189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.139311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.139588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.139733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.139994] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.140182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.140281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.140516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.140952] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.140952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.141131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.141258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.141396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.141540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.141791] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.141950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.142054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.142254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.142384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.142618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.142765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.142879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.143080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.143212] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  592.143448] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f29dc18f10ab4667ad0a0e9f7187c730 in queue reply_156db6c4256a4c58884afce15e6f5340
[  592.149907] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f29dc18f10ab4667ad0a0e9f7187c730
[  592.283783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.283985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.284136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.284402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.284550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.284852] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.285034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.285158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.285379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.285630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.285806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.285937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.286150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.286298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.286449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.286700] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.286856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.286974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.287196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.287333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.287572] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.287723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.287874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.288102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.288238] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  592.288465] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg eaca3d0b6b79456e98dd12db75d940cf in queue reply_156db6c4256a4c58884afce15e6f5340
[  592.295685] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eaca3d0b6b79456e98dd12db75d940cf
[  592.376027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.376027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.376027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.376027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.376027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.376027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.376027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.376027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.376429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.376429] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.376532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.376782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.376854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.377016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.377128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.377382] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.377539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.377661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.377882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.378017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.378277] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.378407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.378524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.378735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.378876] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  592.379114] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bdb23f3289e946a28f23e317c9c849fa in queue reply_156db6c4256a4c58884afce15e6f5340
[  592.387603] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bdb23f3289e946a28f23e317c9c849fa
[  592.421279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.421801] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.421801] env[60890]: ERROR oslo_messaging.rpc.server 
[  592.484058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.484058] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.484058] env[60890]: ERROR oslo_messaging.rpc.server 
[  592.577802] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-d6ce390a-e3ed-4ffb-af33-6be77969f1b2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.578040] env[60890]: Traceback (most recent call last):
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  592.578101] env[60890]:     timer()
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  592.578101] env[60890]:     cb(*args, **kw)
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  592.578101] env[60890]:     result = function(*args, **kwargs)
[  592.578101] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  592.578101] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  592.578101] env[60890]:     return method(*args, **kwargs)
[  592.578101] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  592.578101] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  592.578101] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  592.578101] env[60890]:     result = self.transport._send(
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  592.578101] env[60890]:     return self._driver.send(target, ctxt, message,
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  592.578101] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  592.578101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  592.578101] env[60890]:     raise result
[  592.578101] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.578101] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.637072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.637403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.637604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.637936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.638178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.638504] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-236 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.638661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.638783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.639013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.639262] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.640029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.640029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.640029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.640029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.640029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-236" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.640291] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.640431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.640551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.640756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.640896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.641130] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cf37b2b7-4cb0-4936-8454-14a07da4afe8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.641282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.641399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.641605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.641743] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cf37b2b7-4cb0-4936-8454-14a07da4afe8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  592.642009] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0cab109043e74ae9adcc8ba8f7e4b8ee in queue reply_156db6c4256a4c58884afce15e6f5340
[  592.648329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.648329] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.648329] env[60890]: ERROR oslo_messaging.rpc.server 
[  592.653552] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0cab109043e74ae9adcc8ba8f7e4b8ee
[  592.764453] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.764705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.764821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.765088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.765237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.765496] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.765658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.765794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.766029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.766280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.766437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.766560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.766774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.766920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.767069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.767320] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.767469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.767847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.767847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.767992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.768185] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.768336] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.768461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.768665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.768802] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  592.769019] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ae5f81bfbc60449baa69637ba63a845e in queue reply_156db6c4256a4c58884afce15e6f5340
[  592.777693] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ae5f81bfbc60449baa69637ba63a845e
[  592.790092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.792030] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.792030] env[60890]: ERROR oslo_messaging.rpc.server 
[  592.829335] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.829550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.829676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.829941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.830084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.830341] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.830494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.830612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.830830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.831071] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.831222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.831338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.831543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.831702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.831893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.832309] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.832899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.833823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.833823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.833823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.833823] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.834094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.834157] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.834399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.834557] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  592.834825] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg eb3c0dffd104403ab22be846303a800f in queue reply_156db6c4256a4c58884afce15e6f5340
[  592.881949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.882465] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  592.882465] env[60890]: ERROR oslo_messaging.rpc.server 
[  592.899209] env[60890]: INFO networking_nsxv3.common.synchronization [req-288306b3-030f-47a0-9fe8-eb7a4ca729ee req-bab04bca-6181-4778-bc64-af32136556e9 tempest-ServerActionsTestOtherB-1179403528 tempest-ServerActionsTestOtherB-1179403528-project-member] Enqueued Resource ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 with Priority: HIGHEST for action security_group_members
[  592.902121] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 with Priority: HIGHEST for action security_group_members
[  592.902121] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.902121] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-0cd25894-1483-4857-b8b7-2f23e1ed9869 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.902121] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  592.902121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  592.902121] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('0cd25894-1483-4857-b8b7-2f23e1ed9869',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  592.902121] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9acad60830c14cfeb703043094474d3c in queue reply_156db6c4256a4c58884afce15e6f5340
[  592.975970] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eb3c0dffd104403ab22be846303a800f
[  592.999340] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9acad60830c14cfeb703043094474d3c
[  593.050279] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.050550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.050603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.050886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.051030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.051286] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-448 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.051443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.051775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.051775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.052059] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.052215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.052330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.052764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.052764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.052869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-448" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.054208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.054208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.054208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.054208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.054208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.054435] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1aa09496-c7e9-4a6a-b46b-590344d3449d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.054554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.054849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.054915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.055062] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1aa09496-c7e9-4a6a-b46b-590344d3449d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  593.055290] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e91909f4cdf24544922bb8e0be539040 in queue reply_156db6c4256a4c58884afce15e6f5340
[  593.062961] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e91909f4cdf24544922bb8e0be539040
[  593.144772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cf37b2b7-4cb0-4936-8454-14a07da4afe8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.145213] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.145213] env[60890]: ERROR oslo_messaging.rpc.server 
[  593.271577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.272176] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.272176] env[60890]: ERROR oslo_messaging.rpc.server 
[  593.338125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.338125] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.338125] env[60890]: ERROR oslo_messaging.rpc.server 
[  593.389452] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.389655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.389781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.390043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.390188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.390440] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.390594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.390717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.390927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.391172] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.391324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.391483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.391649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.391790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.391934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.392204] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.392351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.392465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.392688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.392834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.393152] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.393250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.393333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.393539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.393841] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  593.393906] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bea82d6136904580b8118a792b0d9d3a in queue reply_156db6c4256a4c58884afce15e6f5340
[  593.404200] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.404266] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.404374] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.404588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.404588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.404804] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.404908] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.405006] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.405119] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.405270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.405270] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 0cd25894-1483-4857-b8b7-2f23e1ed9869 already deleted.
[  593.405351] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-0cd25894-1483-4857-b8b7-2f23e1ed9869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.420612] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bea82d6136904580b8118a792b0d9d3a
[  593.513495] env[60890]: INFO networking_nsxv3.common.synchronization [None req-9871d152-1a67-4cf5-b1c1-ca9bc0ac8f10 None None] Enqueued Resource ID: 9dc341f3-53f2-4bf3-b3d4-c1811e564fb5 with Priority: HIGHEST for action port
[  593.513766] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9dc341f3-53f2-4bf3-b3d4-c1811e564fb5 with Priority: HIGHEST for action port
[  593.514402] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.514528] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.514596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.514821] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.514910] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.515059] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-9dc341f3-53f2-4bf3-b3d4-c1811e564fb5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.515155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-9dc341f3-53f2-4bf3-b3d4-c1811e564fb5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.515216] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-9dc341f3-53f2-4bf3-b3d4-c1811e564fb5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.515431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-9dc341f3-53f2-4bf3-b3d4-c1811e564fb5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.515508] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9dc341f3-53f2-4bf3-b3d4-c1811e564fb5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  593.515703] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 57343eae08e442a8a33fc88a0d74f3e8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  593.525022] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 57343eae08e442a8a33fc88a0d74f3e8
[  593.559216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1aa09496-c7e9-4a6a-b46b-590344d3449d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.559942] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.559942] env[60890]: ERROR oslo_messaging.rpc.server 
[  593.566566] env[60890]: INFO networking_nsxv3.common.synchronization [req-fffd2f23-5125-4a8d-a244-eaf5f62faa95 req-02db792f-df35-49d8-a3ee-4bd1aaad74f1 tempest-VolumesAssistedSnapshotsTest-1202454958 tempest-VolumesAssistedSnapshotsTest-1202454958-project-member] Enqueued Resource ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 with Priority: HIGHEST for action security_group_members
[  593.567472] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 with Priority: HIGHEST for action security_group_members
[  593.568412] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.568554] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.568620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.569030] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.569030] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.569121] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-6cb50a73-5b5c-401a-b812-ccab4721d4a6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.569164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.569193] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.569341] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.569497] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.569573] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.569626] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.569754] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.569827] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.569913] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('6cb50a73-5b5c-401a-b812-ccab4721d4a6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  593.570093] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0472363892cb434ca4e20d8f4a1dcbb6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  593.588143] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0472363892cb434ca4e20d8f4a1dcbb6
[  593.712537] env[60890]: INFO networking_nsxv3.common.synchronization [None req-e335016a-2b47-495e-84c8-b06216f82e2b None None] Enqueued Resource ID: d2117d7c-2724-46d6-ac12-10af7b7d891e with Priority: HIGHEST for action port
[  593.713136] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d2117d7c-2724-46d6-ac12-10af7b7d891e with Priority: HIGHEST for action port
[  593.713492] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.713612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.713675] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.713871] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.713949] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.714095] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-d2117d7c-2724-46d6-ac12-10af7b7d891e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.714185] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-d2117d7c-2724-46d6-ac12-10af7b7d891e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.714243] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-d2117d7c-2724-46d6-ac12-10af7b7d891e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.714444] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-d2117d7c-2724-46d6-ac12-10af7b7d891e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.714519] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d2117d7c-2724-46d6-ac12-10af7b7d891e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  593.715022] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 76e321aed8274b8a8347995f04dc233f in queue reply_156db6c4256a4c58884afce15e6f5340
[  593.814422] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 76e321aed8274b8a8347995f04dc233f
[  593.878989] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.879188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.879313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.879576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.879719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.880079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.880168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.880274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.880491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.881018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.881018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.881018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.881205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.881345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.881519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.881751] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.881904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.882020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.882224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.882353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.882592] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.882738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.882852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.883054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.883185] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  593.883419] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ec135fa0089d41fc90df9e591b876f66 in queue reply_156db6c4256a4c58884afce15e6f5340
[  593.886603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.886782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.886906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.887208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.887268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.887510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.887648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.887767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.887975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.888254] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.888415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.888535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.888746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.888890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.889039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.889288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.889445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.889562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.889768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.889904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.890145] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.890298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.890415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.890618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.890752] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  593.891654] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg edceacfcdcee43b49269bd18b3b063ea in queue reply_156db6c4256a4c58884afce15e6f5340
[  593.892567] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ec135fa0089d41fc90df9e591b876f66
[  593.895545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.896053] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  593.896053] env[60890]: ERROR oslo_messaging.rpc.server 
[  593.899023] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg edceacfcdcee43b49269bd18b3b063ea
[  593.924710] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.925207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.925207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.925466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.925605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.925918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.926081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.926197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.926411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.926653] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.926803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.926918] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.927122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.927261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.927404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.927669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.927890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.927945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.928174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.928309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  593.928541] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  593.928698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  593.928817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  593.929203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  593.929203] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  593.929384] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 94d50dee2ba84b16b05b38c77b6aab43 in queue reply_156db6c4256a4c58884afce15e6f5340
[  593.940963] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 94d50dee2ba84b16b05b38c77b6aab43
[  594.017623] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-9dc341f3-53f2-4bf3-b3d4-c1811e564fb5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.017925] env[60890]: Traceback (most recent call last):
[  594.018154] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  594.018154] env[60890]:     timer()
[  594.018154] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  594.018154] env[60890]:     cb(*args, **kw)
[  594.018154] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  594.018154] env[60890]:     result = function(*args, **kwargs)
[  594.018154] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  594.018154] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  594.018154] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  594.018154] env[60890]:     return method(*args, **kwargs)
[  594.018434] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  594.018434] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  594.018434] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  594.018434] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  594.018434] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  594.018434] env[60890]:     result = self.transport._send(
[  594.018434] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  594.018434] env[60890]:     return self._driver.send(target, ctxt, message,
[  594.018434] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  594.018434] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  594.018434] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  594.018434] env[60890]:     raise result
[  594.018434] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.018434] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.073796] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.073796] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.073796] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 6cb50a73-5b5c-401a-b812-ccab4721d4a6 already deleted.
[  594.073796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-6cb50a73-5b5c-401a-b812-ccab4721d4a6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.216533] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-d2117d7c-2724-46d6-ac12-10af7b7d891e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.216772] env[60890]: Traceback (most recent call last):
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  594.216817] env[60890]:     timer()
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  594.216817] env[60890]:     cb(*args, **kw)
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  594.216817] env[60890]:     result = function(*args, **kwargs)
[  594.216817] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  594.216817] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  594.216817] env[60890]:     return method(*args, **kwargs)
[  594.216817] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  594.216817] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  594.216817] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  594.216817] env[60890]:     result = self.transport._send(
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  594.216817] env[60890]:     return self._driver.send(target, ctxt, message,
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  594.216817] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  594.216817] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  594.216817] env[60890]:     raise result
[  594.217379] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.217379] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.380025] env[60890]: INFO networking_nsxv3.common.synchronization [None req-6d4586da-b3ac-4a4e-bd2c-bce4ac3a48f8 None None] Enqueued Resource ID: e66eecd1-e426-4165-89d5-f774c9a0cb52 with Priority: HIGHEST for action port
[  594.380290] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e66eecd1-e426-4165-89d5-f774c9a0cb52 with Priority: HIGHEST for action port
[  594.380889] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.381020] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.381089] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.381293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.381376] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.381524] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-e66eecd1-e426-4165-89d5-f774c9a0cb52 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.381618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-e66eecd1-e426-4165-89d5-f774c9a0cb52" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.381680] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-e66eecd1-e426-4165-89d5-f774c9a0cb52" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.381884] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-e66eecd1-e426-4165-89d5-f774c9a0cb52" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.381961] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e66eecd1-e426-4165-89d5-f774c9a0cb52',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  594.382558] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 13b8805ad3984d0681d061fe2e513a4a in queue reply_156db6c4256a4c58884afce15e6f5340
[  594.385984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.387067] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.387067] env[60890]: ERROR oslo_messaging.rpc.server 
[  594.392982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.393401] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.393401] env[60890]: ERROR oslo_messaging.rpc.server 
[  594.406886] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 13b8805ad3984d0681d061fe2e513a4a
[  594.437047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.437047] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.437047] env[60890]: ERROR oslo_messaging.rpc.server 
[  594.445530] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3c2e919adb1240008dadad5f42f5a9d3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  594.476370] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3c2e919adb1240008dadad5f42f5a9d3
[  594.762211] env[60890]: INFO networking_nsxv3.common.synchronization [req-36f80583-9b15-43cc-879a-36b63f3d0025 req-243445aa-ae11-4056-83cb-e8e99a4345ed tempest-ServersWithSpecificFlavorTestJSON-1300140250 tempest-ServersWithSpecificFlavorTestJSON-1300140250-project-member] Enqueued Resource ID: a3a01724-483b-40cb-9132-f26a2a638504 with Priority: HIGHEST for action security_group_members
[  594.762713] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a3a01724-483b-40cb-9132-f26a2a638504 with Priority: HIGHEST for action security_group_members
[  594.763021] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.763139] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.763209] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.763405] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.763482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.763674] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-a3a01724-483b-40cb-9132-f26a2a638504 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.763793] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.763793] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.763936] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.764114] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.764219] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.764296] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.764423] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.764501] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.764585] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('a3a01724-483b-40cb-9132-f26a2a638504',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  594.764781] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 57a1f1b21e5f4f4faf215ab3eb87c651 in queue reply_156db6c4256a4c58884afce15e6f5340
[  594.775637] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 57a1f1b21e5f4f4faf215ab3eb87c651
[  594.888970] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-e66eecd1-e426-4165-89d5-f774c9a0cb52" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.888970] env[60890]: Traceback (most recent call last):
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  594.888970] env[60890]:     timer()
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  594.888970] env[60890]:     cb(*args, **kw)
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  594.888970] env[60890]:     result = function(*args, **kwargs)
[  594.888970] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  594.888970] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  594.888970] env[60890]:     return method(*args, **kwargs)
[  594.888970] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  594.888970] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  594.888970] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  594.888970] env[60890]:     result = self.transport._send(
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  594.888970] env[60890]:     return self._driver.send(target, ctxt, message,
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  594.888970] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  594.888970] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  594.888970] env[60890]:     raise result
[  594.888970] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  594.888970] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  594.904030] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.904440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.904440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.904630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.904808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.905077] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.905248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.905373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.905599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.905844] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.906017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.906116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.906324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.906469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.906619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.906879] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.907050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.907165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.907370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.907704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.907763] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.907879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.908029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.908393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.908393] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  594.908612] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 47c0f107cfec4a7e838ea59a2194041d in queue reply_156db6c4256a4c58884afce15e6f5340
[  594.911043] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.911239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.911378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.911596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.915736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.915972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.916181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.916292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.916518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.918254] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.918254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.918254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.918254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.918254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.918254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.918254] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.918254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.918254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.918254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.918632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  594.918632] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  594.918772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  594.918889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  594.919095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  594.919229] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  594.919461] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fb1b9ac70f0f4b9485f4a4c9521710a6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  594.920682] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 47c0f107cfec4a7e838ea59a2194041d
[  594.929119] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fb1b9ac70f0f4b9485f4a4c9521710a6
[  595.225574] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.225847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.225974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.226245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.226531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.226676] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.226835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.226953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.227339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.227614] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.227817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.227949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.228297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.228450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.228604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.229719] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.229719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.229719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.229719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.229719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.229719] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.229979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.229979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.231877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.231877] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  595.231877] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8df291d992944094b9641f186ba3b102 in queue reply_156db6c4256a4c58884afce15e6f5340
[  595.235950] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8df291d992944094b9641f186ba3b102
[  595.264943] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.265237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.265296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.265539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.265691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.265955] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.266113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.266232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.266447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.266694] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.266851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.266969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.267199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.267334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.267483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.267734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.267891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.268025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.268256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.268393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.268633] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.268786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.268902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.269110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.269243] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  595.269477] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 729201a3bec248e2a3a0feab15c040ef in queue reply_156db6c4256a4c58884afce15e6f5340
[  595.270421] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.270533] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.270601] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.270753] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.270837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.271004] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.271099] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.271160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.271299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.271375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.271456] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: a3a01724-483b-40cb-9132-f26a2a638504 already deleted.
[  595.271535] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-a3a01724-483b-40cb-9132-f26a2a638504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.274447] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 729201a3bec248e2a3a0feab15c040ef
[  595.422837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.423352] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.423352] env[60890]: ERROR oslo_messaging.rpc.server 
[  595.436256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.437030] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.437030] env[60890]: ERROR oslo_messaging.rpc.server 
[  595.735548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.736131] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.736131] env[60890]: ERROR oslo_messaging.rpc.server 
[  595.783503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.783503] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.783503] env[60890]: ERROR oslo_messaging.rpc.server 
[  595.943817] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.944042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.944184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.944452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.944595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.944874] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.945010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.945136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.945353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.945601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.945794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.945860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.946068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.946204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.946351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.946598] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.946749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.946869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.947071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.947199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.947438] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.947590] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.947705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.947928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.948053] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  595.948286] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 18bab0ecd7864bb18649059f48083396 in queue reply_156db6c4256a4c58884afce15e6f5340
[  595.953671] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 18bab0ecd7864bb18649059f48083396
[  595.985017] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.985198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.985330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.985591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.985742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.986006] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.986161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.986280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.986496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.986871] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.987076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.987198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.987407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.987549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.987698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.987953] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.988124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.988244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.988450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.988587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.988830] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.988985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.989104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.989303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.989439] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  595.989665] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 81ccb94c7be0496b84a19be5cc6cc49e in queue reply_156db6c4256a4c58884afce15e6f5340
[  595.991850] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.992312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.992312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.992447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.992585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.992819] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.992971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.993088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.993287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.993716] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.993716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.993886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.993994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.994127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.994593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.996435] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.996435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.996435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.996435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.996435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.996435] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  595.996435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  595.996435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  595.996435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  595.996435] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  595.996435] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 63732fe2dac943808aa33ef45c2c903b in queue reply_156db6c4256a4c58884afce15e6f5340
[  595.997166] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 81ccb94c7be0496b84a19be5cc6cc49e
[  595.997659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.998078] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  595.998078] env[60890]: ERROR oslo_messaging.rpc.server 
[  596.000899] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 63732fe2dac943808aa33ef45c2c903b
[  596.035080] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.035281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.035411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.035777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.035850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.036096] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.036254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.036375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.036588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.036828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.036983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.037098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.037301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.037443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.037588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.037835] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.037990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.038107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.038312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.038444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.038678] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.038831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.038946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.039148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.039282] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  596.039507] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 103f0a550a54431fb171449d1fc38e40 in queue reply_156db6c4256a4c58884afce15e6f5340
[  596.046176] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 103f0a550a54431fb171449d1fc38e40
[  596.122025] env[60890]: INFO networking_nsxv3.common.synchronization [req-a152a8bc-ae1d-440e-a674-03fac5538482 req-a70b2b90-bd57-49aa-88c6-1e0087b3e5de tempest-SecurityGroupRulesTestJSON-432430954 tempest-SecurityGroupRulesTestJSON-432430954-project-member] Enqueued Resource ID: bab379cf-cfcc-4c2e-88e4-2dd08435959f with Priority: HIGHEST for action security_group_rules
[  596.122025] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: bab379cf-cfcc-4c2e-88e4-2dd08435959f with Priority: HIGHEST for action security_group_rules
[  596.122025] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.122025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.122025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.122025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.122025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.122025] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-bab379cf-cfcc-4c2e-88e4-2dd08435959f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.122025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-bab379cf-cfcc-4c2e-88e4-2dd08435959f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.122025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-bab379cf-cfcc-4c2e-88e4-2dd08435959f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.122373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-bab379cf-cfcc-4c2e-88e4-2dd08435959f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.122373] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('bab379cf-cfcc-4c2e-88e4-2dd08435959f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  596.122432] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ecc6a20d0bad4fb1a02fb270dd38530c in queue reply_156db6c4256a4c58884afce15e6f5340
[  596.139670] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ecc6a20d0bad4fb1a02fb270dd38530c
[  596.449987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  596.450513] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  596.450513] env[60890]: ERROR oslo_messaging.rpc.server 
[  596.469727] env[60890]: INFO networking_nsxv3.common.synchronization [None req-08c0d68b-9f05-49bd-becb-03bdf47c496a None None] Enqueued Resource ID: 8dd47c0c-1f45-4b75-9938-2a1d95c1a8de with Priority: HIGHEST for action port
[  596.470272] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8dd47c0c-1f45-4b75-9938-2a1d95c1a8de with Priority: HIGHEST for action port
[  596.470602] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.470726] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.470789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.470993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.471080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.471228] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-8dd47c0c-1f45-4b75-9938-2a1d95c1a8de taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.471322] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-8dd47c0c-1f45-4b75-9938-2a1d95c1a8de" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.471382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-8dd47c0c-1f45-4b75-9938-2a1d95c1a8de" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.471581] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-8dd47c0c-1f45-4b75-9938-2a1d95c1a8de" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.471658] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('8dd47c0c-1f45-4b75-9938-2a1d95c1a8de',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  596.471833] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2f5184e1f076497db6b1f59086265134 in queue reply_156db6c4256a4c58884afce15e6f5340
[  596.477187] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2f5184e1f076497db6b1f59086265134
[  596.655837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  596.656355] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  596.656355] env[60890]: ERROR oslo_messaging.rpc.server 
[  596.657650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  596.658022] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  596.658022] env[60890]: ERROR oslo_messaging.rpc.server 
[  596.660616] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.660768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.660894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.661407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.661407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.661512] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-200 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.661669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.661788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.662002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.662243] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.662397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.662513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.662751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.662897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.663044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-200" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.663293] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.663445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.663568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.665482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.665623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.665885] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-17102ac0-8668-4ecd-ab24-5718992cc0f7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.666053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.666207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.666377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.666511] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17102ac0-8668-4ecd-ab24-5718992cc0f7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  596.666932] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 21be21d2bf2f4ed8b040d4f71868acdf in queue reply_156db6c4256a4c58884afce15e6f5340
[  596.679007] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 21be21d2bf2f4ed8b040d4f71868acdf
[  596.684394] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  596.684548] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  596.684595] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  596.684838] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  596.685107] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.685107] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: bab379cf-cfcc-4c2e-88e4-2dd08435959f already deleted.
[  596.685107] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-bab379cf-cfcc-4c2e-88e4-2dd08435959f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.973516] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-8dd47c0c-1f45-4b75-9938-2a1d95c1a8de" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  596.973782] env[60890]: Traceback (most recent call last):
[  596.973782] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  596.973782] env[60890]:     timer()
[  596.973782] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  596.973782] env[60890]:     cb(*args, **kw)
[  596.973782] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  596.973782] env[60890]:     result = function(*args, **kwargs)
[  596.973782] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  596.973782] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  596.973782] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  596.973782] env[60890]:     return method(*args, **kwargs)
[  596.973782] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  596.973782] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  596.974124] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  596.974124] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  596.974124] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  596.974124] env[60890]:     result = self.transport._send(
[  596.974124] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  596.974124] env[60890]:     return self._driver.send(target, ctxt, message,
[  596.974124] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  596.974124] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  596.974124] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  596.974124] env[60890]:     raise result
[  596.974124] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  596.974124] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.089294] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.089487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.089617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.089907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.090061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.090317] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.090474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.090594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.090812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.091056] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.091211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.091344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.091555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.091698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.091847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.092116] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.092274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.092393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.092605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.092735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.093137] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.093137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.093251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.093458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.093592] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  597.093817] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6a39990d319c44c7a8487ef8498016bf in queue reply_156db6c4256a4c58884afce15e6f5340
[  597.099865] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6a39990d319c44c7a8487ef8498016bf
[  597.169388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-17102ac0-8668-4ecd-ab24-5718992cc0f7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  597.169388] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.169388] env[60890]: ERROR oslo_messaging.rpc.server 
[  597.178718] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.178942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.179074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.179339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.179484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.179737] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.179893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.180030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.180232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.180553] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.180714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.180793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.180960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.181116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.181297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.181506] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.181865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.181865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.182346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.182346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.182346] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.182531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.182651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.182856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.182992] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  597.183221] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 07ca0efd53314c82865abcf3400e7467 in queue reply_156db6c4256a4c58884afce15e6f5340
[  597.188051] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 07ca0efd53314c82865abcf3400e7467
[  597.236769] env[60890]: INFO networking_nsxv3.common.synchronization [None req-8ea945d3-4ed4-47c4-9236-039112829cb8 tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 with Priority: HIGHEST for action security_group_rules
[  597.237273] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 with Priority: HIGHEST for action security_group_rules
[  597.237586] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.237710] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.237774] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.237999] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.238084] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.238240] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.238334] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.238398] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.238590] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.238668] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  597.238843] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 900e844f2db542fcb7fcf15d05c11bfe in queue reply_156db6c4256a4c58884afce15e6f5340
[  597.248404] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 900e844f2db542fcb7fcf15d05c11bfe
[  597.357122] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.357375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.357556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.357861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.358399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.358399] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.358525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.358623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.358870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.359154] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.359365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.359515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.359756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.359962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.360539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.360539] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.360539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.360694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.360935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.361106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.361385] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.361565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.361723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.361966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.362131] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  597.362404] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 70b130cc066440dcb9278e5e94dbb112 in queue reply_156db6c4256a4c58884afce15e6f5340
[  597.367863] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 70b130cc066440dcb9278e5e94dbb112
[  597.425063] env[60890]: INFO networking_nsxv3.common.synchronization [None req-17d00933-58c7-45ab-923b-aa9c03eaa523 tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 with Priority: HIGHEST for action security_group_rules
[  597.425834] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 with Priority: HIGHEST for action security_group_rules
[  597.426371] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.426951] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.427375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.427375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.427459] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.427858] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.427915] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.596203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  597.596927] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.596927] env[60890]: ERROR oslo_messaging.rpc.server 
[  597.685419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  597.686249] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.686249] env[60890]: ERROR oslo_messaging.rpc.server 
[  597.759287] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  597.759538] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  597.759637] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.759892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.760000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.761414] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 already deleted.
[  597.761414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.761414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  597.761414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  597.761414] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  597.761414] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a9134556bd4840c7a21290353838cc7e in queue reply_156db6c4256a4c58884afce15e6f5340
[  597.781052] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a9134556bd4840c7a21290353838cc7e
[  597.864904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  597.865448] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  597.865448] env[60890]: ERROR oslo_messaging.rpc.server 
[  598.231797] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.231797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.231797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.231797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.231797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.231797] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.231797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.231797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.231797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.231797] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.231797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.233756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.233756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.233756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.237782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.237782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.237782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.237782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.237782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.237782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.237782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-767e6e62-a97d-495a-85f4-7722190a69d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.237782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.237782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.237782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.237782] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('767e6e62-a97d-495a-85f4-7722190a69d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  598.237782] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ffeb0ba97ebd46ff9f821d7e94200fa3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  598.242622] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ffeb0ba97ebd46ff9f821d7e94200fa3
[  598.281511] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.281897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.282511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.282909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.284089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.284477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.284814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.285099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.285444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.285861] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.286159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.286383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.287637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.287939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.288264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.288721] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.289416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.289617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.290523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.290778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.291244] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.291551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.291774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.292545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.292809] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('24f36f7c-4681-4b9b-bcf3-1dac83ecb968',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  598.293158] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ce135524373446c19eaadda7e1b33a64 in queue reply_156db6c4256a4c58884afce15e6f5340
[  598.297116] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.297354] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.297500] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.299000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.299195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.299357] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 already deleted.
[  598.299644] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.315873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ce135524373446c19eaadda7e1b33a64
[  598.455617] env[60890]: INFO networking_nsxv3.common.synchronization [req-96919bd4-a1dd-4c5b-913f-4a6bd23ee594 req-5f9a1745-8fc5-4f90-93c0-56bc8b4ea680 tempest-AttachInterfacesUnderV243Test-1652000262 tempest-AttachInterfacesUnderV243Test-1652000262-project-member] Enqueued Resource ID: 5856cafc-e8df-48e6-a28b-0e951067688f with Priority: HIGHEST for action security_group_members
[  598.456353] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5856cafc-e8df-48e6-a28b-0e951067688f with Priority: HIGHEST for action security_group_members
[  598.456811] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.457098] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.457098] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.457275] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.457356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.457512] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-5856cafc-e8df-48e6-a28b-0e951067688f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.457595] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.457742] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.457810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.457930] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.458031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.458085] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.458207] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.458265] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.458351] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('5856cafc-e8df-48e6-a28b-0e951067688f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  598.458536] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a62b1af302934339a990cec2f0aabdab in queue reply_156db6c4256a4c58884afce15e6f5340
[  598.467358] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a62b1af302934339a990cec2f0aabdab
[  598.740215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-767e6e62-a97d-495a-85f4-7722190a69d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  598.741068] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  598.741068] env[60890]: ERROR oslo_messaging.rpc.server 
[  598.795903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-24f36f7c-4681-4b9b-bcf3-1dac83ecb968" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  598.796470] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  598.796470] env[60890]: ERROR oslo_messaging.rpc.server 
[  598.828618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.828833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.828973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.829253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.829412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.830045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.830045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.830045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.830267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.830392] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.830546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.830660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.830871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.831018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.831173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.831425] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.831579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.831698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.831913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.832072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.832317] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.832485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.832652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.832892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.833036] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  598.835243] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2fd5a33232b24a7e9cee2d921ad04618 in queue reply_156db6c4256a4c58884afce15e6f5340
[  598.844606] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2fd5a33232b24a7e9cee2d921ad04618
[  598.961098] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.961273] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.961324] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.961542] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.961641] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.961824] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  598.961950] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  598.961983] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  598.962156] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  598.962196] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  598.962269] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 5856cafc-e8df-48e6-a28b-0e951067688f already deleted.
[  598.962350] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-5856cafc-e8df-48e6-a28b-0e951067688f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.234102] env[60890]: INFO networking_nsxv3.common.synchronization [req-3fff35ff-618e-48a3-bb5c-136d8cf1b5e6 req-9519c715-f71c-4a01-9ace-12f1c9bfcbad tempest-ServersAdminTestJSON-603115879 tempest-ServersAdminTestJSON-603115879-project-member] Enqueued Resource ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd with Priority: HIGHEST for action security_group_members
[  599.234102] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd with Priority: HIGHEST for action security_group_members
[  599.234102] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.234102] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.234102] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.234491] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.234523] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.234673] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-abd91d97-a4e5-45fe-9581-3f1337cd29cd taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.234815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.235115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.235115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.235323] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.235636] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.235636] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.235636] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.235782] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.235811] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('abd91d97-a4e5-45fe-9581-3f1337cd29cd',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  599.236057] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7cf655e4aba14361851b956dd9a2072c in queue reply_156db6c4256a4c58884afce15e6f5340
[  599.247212] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7cf655e4aba14361851b956dd9a2072c
[  599.338585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  599.339104] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  599.339104] env[60890]: ERROR oslo_messaging.rpc.server 
[  599.343675] env[60890]: INFO networking_nsxv3.common.synchronization [req-85a55bd4-b87c-4070-b69e-36a38395bf69 req-3ecc777e-6cff-48d7-8e6a-1f7f2ceb58f3 tempest-ServerRescueNegativeTestJSON-1640230806 tempest-ServerRescueNegativeTestJSON-1640230806-project-member] Enqueued Resource ID: e314c230-88be-4118-8948-b5f480080893 with Priority: HIGHEST for action security_group_members
[  599.344186] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e314c230-88be-4118-8948-b5f480080893 with Priority: HIGHEST for action security_group_members
[  599.344487] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.344618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.344799] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.345119] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.345216] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.345383] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e314c230-88be-4118-8948-b5f480080893 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.345485] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.345551] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.345938] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.346090] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.346186] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.346249] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.346393] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.346474] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.346563] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e314c230-88be-4118-8948-b5f480080893',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  599.346734] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0f235153f8ff406fa7ecc8bc18669b18 in queue reply_156db6c4256a4c58884afce15e6f5340
[  599.360602] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0f235153f8ff406fa7ecc8bc18669b18
[  599.546887] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.547285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.547412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.547708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.547870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.548183] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.548353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.548475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.548953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.549661] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.549661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.549661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.549811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.549920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.550083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.550342] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.550498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.550646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.550818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.550953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.551192] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.551344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.551462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.551685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.551827] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  599.552077] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3bf719e2bb474b1abcf8c01683c9d74d in queue reply_156db6c4256a4c58884afce15e6f5340
[  599.560405] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3bf719e2bb474b1abcf8c01683c9d74d
[  599.738227] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.738380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.738447] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.738684] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.738785] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.738986] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.739104] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.739170] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.739395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.739395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.739467] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd already deleted.
[  599.739546] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.741646] env[60890]: INFO networking_nsxv3.common.synchronization [req-a0e093bf-a434-4311-8ef8-7b23964970a1 req-05d216f6-100e-44ec-a63c-c0807ac019a9 tempest-ServerRescueNegativeTestJSON-1640230806 tempest-ServerRescueNegativeTestJSON-1640230806-project-member] Enqueued Resource ID: e314c230-88be-4118-8948-b5f480080893 with Priority: HIGHEST for action security_group_members
[  599.742178] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e314c230-88be-4118-8948-b5f480080893 with Priority: HIGHEST for action security_group_members
[  599.742477] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.742693] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.742810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.742987] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.743072] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.743231] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e314c230-88be-4118-8948-b5f480080893 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.743315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.764381] env[60890]: INFO networking_nsxv3.common.synchronization [req-c439940e-b724-4c5a-bbc9-3fed193a5be5 req-be0445bc-1e4c-4b32-827c-fd05faa30b07 tempest-SecurityGroupRulesTestJSON-432430954 tempest-SecurityGroupRulesTestJSON-432430954-project-member] Enqueued Resource ID: 6175b599-aeed-4d0e-8502-6e5357c4cea1 with Priority: HIGHEST for action security_group_rules
[  599.764912] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6175b599-aeed-4d0e-8502-6e5357c4cea1 with Priority: HIGHEST for action security_group_rules
[  599.765223] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.765344] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.765406] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.765600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.765697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.765853] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-6175b599-aeed-4d0e-8502-6e5357c4cea1 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.765948] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-6175b599-aeed-4d0e-8502-6e5357c4cea1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.766007] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-6175b599-aeed-4d0e-8502-6e5357c4cea1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.766208] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-6175b599-aeed-4d0e-8502-6e5357c4cea1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.766286] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('6175b599-aeed-4d0e-8502-6e5357c4cea1',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  599.766458] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d77c25e7d5d340a298bdb25a290fc7c1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  599.792437] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d77c25e7d5d340a298bdb25a290fc7c1
[  599.849038] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.849238] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.849338] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.849631] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.849820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.850321] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.850321] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.850321] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.850482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.851143] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.851143] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e314c230-88be-4118-8948-b5f480080893 already deleted.
[  599.851143] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.851143] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.851300] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.851868] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  599.852092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  599.852158] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  599.852368] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  599.852486] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  599.852618] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e314c230-88be-4118-8948-b5f480080893',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  599.853269] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 49464bb1085d4a14a7c8e1a6f18bffbb in queue reply_156db6c4256a4c58884afce15e6f5340
[  599.864567] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 49464bb1085d4a14a7c8e1a6f18bffbb
[  600.055824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  600.055824] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  600.055824] env[60890]: ERROR oslo_messaging.rpc.server 
[  600.291045] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.291045] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.291045] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.291045] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.291045] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.291045] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 6175b599-aeed-4d0e-8502-6e5357c4cea1 already deleted.
[  600.291045] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-6175b599-aeed-4d0e-8502-6e5357c4cea1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.361337] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.361337] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.361337] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e314c230-88be-4118-8948-b5f480080893 already deleted.
[  600.361337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.692280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.692280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.692280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.692280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.692280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.692280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.692280] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  600.692280] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0edda8b6236145d88e13705b01898d06 in queue reply_156db6c4256a4c58884afce15e6f5340
[  600.698110] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0edda8b6236145d88e13705b01898d06
[  600.707064] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.707256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.707380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.707633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.707860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.708093] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-768 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.708200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.708315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.708528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.708787] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.708975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.709097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.709300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.709438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.709580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-768" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.709899] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.709983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.710095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.710294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.710422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  600.710705] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  600.710794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  600.710907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  600.711104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  600.711289] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  600.711453] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e71b3cc6f0c948ce950822db1d899aa2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  600.716233] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e71b3cc6f0c948ce950822db1d899aa2
[  601.196645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  601.197206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  601.197206] env[60890]: ERROR oslo_messaging.rpc.server 
[  601.214230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-592ae758-3c9d-4dd8-9ce7-53cdbcd4cfc2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  601.214781] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  601.214781] env[60890]: ERROR oslo_messaging.rpc.server 
[  601.841405] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  601.841676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  601.841750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  601.842028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  601.842176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  601.842431] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  601.842602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  601.842736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  601.842994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  601.843249] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  601.843347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  601.843465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  601.843669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  601.846193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  601.846193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  601.846193] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  601.846193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  601.846193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  601.846193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  601.846193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  601.846532] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  601.846532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  601.846609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  601.847130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  601.847291] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  601.847527] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ed69216d47404afc85b1f4eccc791681 in queue reply_156db6c4256a4c58884afce15e6f5340
[  601.853449] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ed69216d47404afc85b1f4eccc791681
[  602.249067] env[60890]: INFO networking_nsxv3.common.synchronization [req-954f4716-cea8-4ea6-a0b5-c481f4be5ad0 req-6484151c-af16-404c-8551-e2a2a30544ac tempest-ServersAdminTestJSON-603115879 tempest-ServersAdminTestJSON-603115879-project-member] Enqueued Resource ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd with Priority: HIGHEST for action security_group_members
[  602.249582] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd with Priority: HIGHEST for action security_group_members
[  602.249899] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.250025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.250362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.250362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.250362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.250516] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-abd91d97-a4e5-45fe-9581-3f1337cd29cd taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.250611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.250673] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.250820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.250952] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.251042] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.251103] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.251661] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.251777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.251873] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('abd91d97-a4e5-45fe-9581-3f1337cd29cd',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  602.252076] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8d9d9484a4054bd7ba92c7cb38648013 in queue reply_156db6c4256a4c58884afce15e6f5340
[  602.261164] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8d9d9484a4054bd7ba92c7cb38648013
[  602.349958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  602.350568] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  602.350568] env[60890]: ERROR oslo_messaging.rpc.server 
[  602.754099] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.754280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.754361] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.754580] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.754708] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.754902] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.755001] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.755066] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.755231] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.755290] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.755373] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: abd91d97-a4e5-45fe-9581-3f1337cd29cd already deleted.
[  602.755450] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-abd91d97-a4e5-45fe-9581-3f1337cd29cd" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.964041] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.964041] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.964041] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.964041] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.964041] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.964041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.964041] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  602.965146] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5464523fc6184128ab9323f017a12cfc in queue reply_156db6c4256a4c58884afce15e6f5340
[  602.970375] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5464523fc6184128ab9323f017a12cfc
[  602.978540] env[60890]: INFO networking_nsxv3.common.synchronization [req-1a117e94-0fed-4ac4-84a1-d89a6caad47e req-eb815966-519d-4270-b0e5-f60a26788c82 tempest-SecurityGroupRulesTestJSON-432430954 tempest-SecurityGroupRulesTestJSON-432430954-project-member] Enqueued Resource ID: 7ad28751-a1fd-4212-a59e-f90142543b34 with Priority: HIGHEST for action security_group_rules
[  602.979160] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7ad28751-a1fd-4212-a59e-f90142543b34 with Priority: HIGHEST for action security_group_rules
[  602.979485] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.979653] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.979722] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.979901] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.979982] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  602.980142] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-7ad28751-a1fd-4212-a59e-f90142543b34 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  602.980274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-7ad28751-a1fd-4212-a59e-f90142543b34" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  602.980305] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-7ad28751-a1fd-4212-a59e-f90142543b34" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  602.980484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-7ad28751-a1fd-4212-a59e-f90142543b34" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  602.980572] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('7ad28751-a1fd-4212-a59e-f90142543b34',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  602.980729] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5283c4d0c18948cab6e681176ce6eb84 in queue reply_156db6c4256a4c58884afce15e6f5340
[  602.996079] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5283c4d0c18948cab6e681176ce6eb84
[  603.466759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  603.467282] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  603.467282] env[60890]: ERROR oslo_messaging.rpc.server 
[  603.502537] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  603.502726] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  603.502800] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  603.503013] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  603.503107] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  603.503200] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 7ad28751-a1fd-4212-a59e-f90142543b34 already deleted.
[  603.503283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-7ad28751-a1fd-4212-a59e-f90142543b34" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  603.613729] env[60890]: INFO networking_nsxv3.common.synchronization [None req-49386c25-0092-4f79-8dce-cfc02b3cc0a3 None None] Enqueued Resource ID: e1ad077a-48d8-4512-a46d-1868bb5a6ede with Priority: HIGHEST for action port
[  603.614018] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e1ad077a-48d8-4512-a46d-1868bb5a6ede with Priority: HIGHEST for action port
[  603.614620] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  603.614775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  603.614826] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  603.616100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  603.616100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  603.616100] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-e1ad077a-48d8-4512-a46d-1868bb5a6ede taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  603.616100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-e1ad077a-48d8-4512-a46d-1868bb5a6ede" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  603.616100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-e1ad077a-48d8-4512-a46d-1868bb5a6ede" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  603.616100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-e1ad077a-48d8-4512-a46d-1868bb5a6ede" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  603.616100] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e1ad077a-48d8-4512-a46d-1868bb5a6ede',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  603.616100] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b431c560beee4fbda30546a4f712e470 in queue reply_156db6c4256a4c58884afce15e6f5340
[  603.621189] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b431c560beee4fbda30546a4f712e470
[  604.119046] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-e1ad077a-48d8-4512-a46d-1868bb5a6ede" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.119046] env[60890]: Traceback (most recent call last):
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  604.119046] env[60890]:     timer()
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  604.119046] env[60890]:     cb(*args, **kw)
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  604.119046] env[60890]:     result = function(*args, **kwargs)
[  604.119046] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  604.119046] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  604.119046] env[60890]:     return method(*args, **kwargs)
[  604.119046] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  604.119046] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  604.119046] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  604.119046] env[60890]:     result = self.transport._send(
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  604.119046] env[60890]:     return self._driver.send(target, ctxt, message,
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  604.119046] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  604.119046] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  604.119046] env[60890]:     raise result
[  604.119046] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  604.119046] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  604.398094] env[60890]: INFO networking_nsxv3.common.synchronization [req-d6583788-6091-445e-b904-99edd86ac735 req-1fd5bbf1-2821-4221-beb1-1d266978e5fb tempest-DeleteServersAdminTestJSON-570518421 tempest-DeleteServersAdminTestJSON-570518421-project-member] Enqueued Resource ID: 86656952-d254-40e4-a34d-b4f64574ede0 with Priority: HIGHEST for action security_group_members
[  604.398715] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 86656952-d254-40e4-a34d-b4f64574ede0 with Priority: HIGHEST for action security_group_members
[  604.399101] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.399254] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.399331] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.399542] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.399631] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.399788] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-86656952-d254-40e4-a34d-b4f64574ede0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.399887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.399948] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.400165] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.400361] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.400408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.400450] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.400591] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.400672] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.400763] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('86656952-d254-40e4-a34d-b4f64574ede0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  604.400937] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4d0a63d819a3429e8cc4f4c6701561c9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  604.417431] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4d0a63d819a3429e8cc4f4c6701561c9
[  604.469309] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.469514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.469640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.469912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.470073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.470511] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.470511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.470638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.470816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.471131] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.471297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.471414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.471624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.471764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.471910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.472328] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.472511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.472613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.472827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.472959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.473317] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.473490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.473609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.473819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.473967] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  604.474205] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9078e27fbf86418c9d44809f6d45f378 in queue reply_156db6c4256a4c58884afce15e6f5340
[  604.480138] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9078e27fbf86418c9d44809f6d45f378
[  604.902991] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.903230] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.903263] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.903473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.903570] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.903761] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  604.903858] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  604.903922] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  604.904091] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  604.904172] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.904259] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 86656952-d254-40e4-a34d-b4f64574ede0 already deleted.
[  604.904336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.976661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  604.977210] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  604.977210] env[60890]: ERROR oslo_messaging.rpc.server 
[  605.371297] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.371297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.371297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.371297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.371297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.371677] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.371677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.371677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.373849] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.373849] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.373849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.373849] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.374380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.374380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.374380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.374681] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  605.374803] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e638a7a296484c7fa59bf3c195ed4788 in queue reply_156db6c4256a4c58884afce15e6f5340
[  605.384816] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e638a7a296484c7fa59bf3c195ed4788
[  605.477257] env[60890]: INFO networking_nsxv3.common.synchronization [req-04e48f95-01bb-4fb5-97c1-e4d876dae3cc req-2ed7ca9b-0e24-40b0-92fb-874760976899 tempest-SecurityGroupRulesTestJSON-432430954 tempest-SecurityGroupRulesTestJSON-432430954-project-member] Enqueued Resource ID: 9a3213bc-84ba-4ccd-b71a-7f90c14317e2 with Priority: HIGHEST for action security_group_rules
[  605.477519] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9a3213bc-84ba-4ccd-b71a-7f90c14317e2 with Priority: HIGHEST for action security_group_rules
[  605.478062] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.478190] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.478260] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.478463] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.478550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.478717] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-9a3213bc-84ba-4ccd-b71a-7f90c14317e2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.478817] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-9a3213bc-84ba-4ccd-b71a-7f90c14317e2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.478878] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-9a3213bc-84ba-4ccd-b71a-7f90c14317e2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.479083] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-9a3213bc-84ba-4ccd-b71a-7f90c14317e2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.479160] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('9a3213bc-84ba-4ccd-b71a-7f90c14317e2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  605.479335] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7d5342d9892c40bdbf41d3625c672a8c in queue reply_156db6c4256a4c58884afce15e6f5340
[  605.488819] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7d5342d9892c40bdbf41d3625c672a8c
[  605.743645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.743859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.743985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.744279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.744422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.744932] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.745132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.745185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.745408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.745743] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.745990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.746122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.746350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.746504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.746658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.746943] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.747112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.747233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.747454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.747610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.747838] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.747996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.748129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.748338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.748476] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  605.748710] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 769df5d5baac49658c32fbfe1ed78975 in queue reply_156db6c4256a4c58884afce15e6f5340
[  605.848090] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 769df5d5baac49658c32fbfe1ed78975
[  605.850037] env[60890]: INFO networking_nsxv3.common.synchronization [None req-6ec0c63d-97e3-42e0-bcce-ef5d3f1b05fd None None] Enqueued Resource ID: 09b0abeb-2c01-4a11-9c6f-807df679cb4e with Priority: HIGHEST for action port
[  605.850505] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 09b0abeb-2c01-4a11-9c6f-807df679cb4e with Priority: HIGHEST for action port
[  605.851032] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.851156] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.851216] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.851417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.851495] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.851637] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-09b0abeb-2c01-4a11-9c6f-807df679cb4e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.851729] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-09b0abeb-2c01-4a11-9c6f-807df679cb4e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.851790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-09b0abeb-2c01-4a11-9c6f-807df679cb4e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.851982] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-09b0abeb-2c01-4a11-9c6f-807df679cb4e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.852070] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('09b0abeb-2c01-4a11-9c6f-807df679cb4e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  605.852242] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 51a29ba8ea6844a4b217af149314f4e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  605.862428] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 51a29ba8ea6844a4b217af149314f4e0
[  605.882053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  605.883262] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  605.883262] env[60890]: ERROR oslo_messaging.rpc.server 
[  605.999420] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  605.999571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  605.999643] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  605.999868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  605.999949] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.000407] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 9a3213bc-84ba-4ccd-b71a-7f90c14317e2 already deleted.
[  606.000407] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-9a3213bc-84ba-4ccd-b71a-7f90c14317e2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.033920] env[60890]: INFO networking_nsxv3.common.synchronization [None req-b07f061b-e718-469e-8fdc-9fafca99859e tempest-AttachVolumeTestJSON-389769224 tempest-AttachVolumeTestJSON-389769224-project-member] Enqueued Resource ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 with Priority: HIGHEST for action security_group_rules
[  606.034455] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 with Priority: HIGHEST for action security_group_rules
[  606.034836] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.034965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.035027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.035226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.035309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.035455] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.035550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.035610] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.036101] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.036196] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('73956ef6-bfe0-49e9-80aa-feabeac5ab67',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  606.036361] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6b4eb173d5e9430298543358cb2668e8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  606.045373] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6b4eb173d5e9430298543358cb2668e8
[  606.250869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  606.251385] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  606.251385] env[60890]: ERROR oslo_messaging.rpc.server 
[  606.305496] env[60890]: INFO networking_nsxv3.common.synchronization [None req-ba44029f-abfe-4546-9b85-8afb3251a402 tempest-AttachVolumeTestJSON-389769224 tempest-AttachVolumeTestJSON-389769224-project-member] Enqueued Resource ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 with Priority: HIGHEST for action security_group_rules
[  606.306554] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 with Priority: HIGHEST for action security_group_rules
[  606.306879] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.307036] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.307116] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.307317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.307401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.307560] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.307643] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.356561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-09b0abeb-2c01-4a11-9c6f-807df679cb4e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.356862] env[60890]: Traceback (most recent call last):
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  606.356959] env[60890]:     timer()
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  606.356959] env[60890]:     cb(*args, **kw)
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  606.356959] env[60890]:     result = function(*args, **kwargs)
[  606.356959] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  606.356959] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  606.356959] env[60890]:     return method(*args, **kwargs)
[  606.356959] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  606.356959] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  606.356959] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  606.356959] env[60890]:     result = self.transport._send(
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  606.356959] env[60890]:     return self._driver.send(target, ctxt, message,
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  606.356959] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  606.356959] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  606.356959] env[60890]:     raise result
[  606.356959] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  606.356959] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  606.555902] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.556207] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.556207] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.556358] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.556451] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.556554] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 already deleted.
[  606.556625] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.556774] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.556937] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.557041] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('73956ef6-bfe0-49e9-80aa-feabeac5ab67',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  606.557261] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7cc8014157874a549afa520283c301fd in queue reply_156db6c4256a4c58884afce15e6f5340
[  606.572528] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7cc8014157874a549afa520283c301fd
[  606.965984] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.966187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.966301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.966637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.966745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.967206] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.967206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.967320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.967456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.967744] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.967905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.968037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.968251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.968396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.968549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.968911] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.969012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.969136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.969345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.969482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  606.969724] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  606.969964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  606.970138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  606.971050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  606.971215] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  606.971459] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 65487311bf484b4da525415c317b7c4b in queue reply_156db6c4256a4c58884afce15e6f5340
[  606.977851] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 65487311bf484b4da525415c317b7c4b
[  607.071547] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  607.071547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  607.071547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  607.071547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  607.071547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  607.072221] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  607.072323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  607.072494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  607.072915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  607.072915] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  607.073080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  607.073170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  607.073374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  607.073514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  607.073658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  607.073899] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  607.074060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  607.074162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  607.074356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  607.074483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  607.074734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  607.074891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  607.075034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  607.075434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  607.075434] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  607.075629] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 82b6ec005a874f888900097785afb472 in queue reply_156db6c4256a4c58884afce15e6f5340
[  607.077526] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  607.077656] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  607.077699] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  607.077848] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  607.077932] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  607.078017] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 already deleted.
[  607.078095] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  607.082892] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 82b6ec005a874f888900097785afb472
[  607.474507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  607.474507] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  607.474507] env[60890]: ERROR oslo_messaging.rpc.server 
[  607.579755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  607.580297] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  607.580297] env[60890]: ERROR oslo_messaging.rpc.server 
[  608.105318] env[60890]: INFO networking_nsxv3.common.synchronization [req-37254212-235a-4f54-8b49-ff832d0ee34b req-2ea84f0b-0e86-4956-9e33-6dad1bcd0d3b tempest-SecurityGroupRulesTestJSON-432430954 tempest-SecurityGroupRulesTestJSON-432430954-project-member] Enqueued Resource ID: a8165742-514c-424f-aeed-5bf70ad8974d with Priority: HIGHEST for action security_group_rules
[  608.105318] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a8165742-514c-424f-aeed-5bf70ad8974d with Priority: HIGHEST for action security_group_rules
[  608.105318] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.105318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.105318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.105318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.105318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.105318] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-a8165742-514c-424f-aeed-5bf70ad8974d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.105318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.105318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.105318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.105318] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('a8165742-514c-424f-aeed-5bf70ad8974d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  608.105318] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg af25a59134944a31a7488c7570659738 in queue reply_156db6c4256a4c58884afce15e6f5340
[  608.161998] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.162210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.162340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.162612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.162783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.163020] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.163205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.163325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.164935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.164935] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.164935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.164935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.164935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.164935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.165160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.166579] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.166579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.167674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.169613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.169613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.169613] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.169613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.169613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.169613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.169613] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  608.169613] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 78d7815b29d84af98bb39e6d2e5d46e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  608.187136] env[60890]: INFO networking_nsxv3.common.synchronization [None req-d362a6e6-8038-4d66-906f-80c311abf1f8 None None] Enqueued Resource ID: 8cc4deb5-ad28-40b6-9e14-b5b967cba23c with Priority: HIGHEST for action port
[  608.187136] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg af25a59134944a31a7488c7570659738
[  608.187136] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8cc4deb5-ad28-40b6-9e14-b5b967cba23c with Priority: HIGHEST for action port
[  608.187136] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.187136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.187136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.187136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.187136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.187136] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-8cc4deb5-ad28-40b6-9e14-b5b967cba23c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.187136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-8cc4deb5-ad28-40b6-9e14-b5b967cba23c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.187136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-8cc4deb5-ad28-40b6-9e14-b5b967cba23c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.187136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-8cc4deb5-ad28-40b6-9e14-b5b967cba23c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.187136] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('8cc4deb5-ad28-40b6-9e14-b5b967cba23c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  608.187136] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6d3ad4bffbe4485eba1bcbbbf4ff3285 in queue reply_156db6c4256a4c58884afce15e6f5340
[  608.187797] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 78d7815b29d84af98bb39e6d2e5d46e0
[  608.217623] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6d3ad4bffbe4485eba1bcbbbf4ff3285
[  608.333103] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.333314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.333440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.333707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.333847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.334095] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.334246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.334363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.334575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.334840] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.334991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.335107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.335312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.335451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.335597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.335914] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.336097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.336217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.336426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.336557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.336847] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.336941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.337057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.337260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.337392] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  608.337620] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 88d272356d954a1fa73f4b2d8a22ab31 in queue reply_156db6c4256a4c58884afce15e6f5340
[  608.404770] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 88d272356d954a1fa73f4b2d8a22ab31
[  608.625855] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.625855] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.625855] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.625855] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.625855] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.625855] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: a8165742-514c-424f-aeed-5bf70ad8974d already deleted.
[  608.625855] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.676089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  608.676089] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  608.676089] env[60890]: ERROR oslo_messaging.rpc.server 
[  608.690003] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-8cc4deb5-ad28-40b6-9e14-b5b967cba23c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.690252] env[60890]: Traceback (most recent call last):
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  608.690294] env[60890]:     timer()
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  608.690294] env[60890]:     cb(*args, **kw)
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  608.690294] env[60890]:     result = function(*args, **kwargs)
[  608.690294] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  608.690294] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  608.690294] env[60890]:     return method(*args, **kwargs)
[  608.690294] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  608.690294] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  608.690294] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  608.690294] env[60890]:     result = self.transport._send(
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  608.690294] env[60890]:     return self._driver.send(target, ctxt, message,
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  608.690294] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  608.690294] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  608.690294] env[60890]:     raise result
[  608.690854] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  608.690854] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  608.840083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  608.840615] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  608.840615] env[60890]: ERROR oslo_messaging.rpc.server 
[  608.859794] env[60890]: INFO networking_nsxv3.common.synchronization [req-fb3647e4-7474-4464-862d-f4d04e5155a1 req-f8ec07a7-e93e-49eb-84ba-2c326d1bbc6e tempest-SecurityGroupRulesTestJSON-432430954 tempest-SecurityGroupRulesTestJSON-432430954-project-member] Enqueued Resource ID: a8165742-514c-424f-aeed-5bf70ad8974d with Priority: HIGHEST for action security_group_rules
[  608.860351] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a8165742-514c-424f-aeed-5bf70ad8974d with Priority: HIGHEST for action security_group_rules
[  608.860680] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.860809] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.860877] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.861078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.861163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  608.861347] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-a8165742-514c-424f-aeed-5bf70ad8974d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  608.861479] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  608.861560] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  608.861693] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  608.861773] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('a8165742-514c-424f-aeed-5bf70ad8974d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  608.861959] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 66efe69700524ff89ea298ef849e0a05 in queue reply_156db6c4256a4c58884afce15e6f5340
[  608.877424] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 66efe69700524ff89ea298ef849e0a05
[  609.241588] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.241793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.241887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.242160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.242334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.242596] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.242753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.242875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.243094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.243336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.246495] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.246495] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.246495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.246495] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  609.246495] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4ea64a7a6ecc4a6692cfd74259ca5410 in queue reply_156db6c4256a4c58884afce15e6f5340
[  609.251205] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4ea64a7a6ecc4a6692cfd74259ca5410
[  609.383764] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.383917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.383989] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.384231] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.384363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.384420] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: a8165742-514c-424f-aeed-5bf70ad8974d already deleted.
[  609.384501] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.408794] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.409010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.409139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.409398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.409556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.409818] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.409978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.410095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.410307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.410551] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.410707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.410826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.411037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.411180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.411325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.411570] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.411716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.411827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.413035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.413249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.413510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.413674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.413795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.414012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.414151] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  609.414382] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0fa99df7e5ee42ee96279b8ffcc3700c in queue reply_156db6c4256a4c58884afce15e6f5340
[  609.419143] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0fa99df7e5ee42ee96279b8ffcc3700c
[  609.465956] env[60890]: INFO networking_nsxv3.common.synchronization [req-1b1d843e-1b51-4030-b1d1-4b2e0d4e78c0 req-ef16f79a-1fe0-40ab-a2f4-82fe89159e82 tempest-ListServersNegativeTestJSON-955562437 tempest-ListServersNegativeTestJSON-955562437-project-member] Enqueued Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  609.465956] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  609.465956] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.465956] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-976f3f29-8ecd-43ae-ac0b-7b7127db7112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.465956] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.465956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.465956] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('976f3f29-8ecd-43ae-ac0b-7b7127db7112',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  609.465956] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ff44cb23763d497e91fff23fac1af7b8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  609.476451] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ff44cb23763d497e91fff23fac1af7b8
[  609.748875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  609.749373] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  609.749373] env[60890]: ERROR oslo_messaging.rpc.server 
[  609.918946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  609.918946] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  609.918946] env[60890]: ERROR oslo_messaging.rpc.server 
[  609.968086] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.968086] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  609.968086] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 already deleted.
[  609.968086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.357728] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.357728] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.357728] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.357728] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.357728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.360652] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.360652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.360652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.360652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.360652] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  610.360652] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8691a4ccac21437e8757ce4024a443a1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  610.365644] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8691a4ccac21437e8757ce4024a443a1
[  610.389403] env[60890]: INFO networking_nsxv3.common.synchronization [None req-ed0780b1-ffea-4103-a250-26f701c391bc None None] Enqueued Resource ID: d51d8d24-4b4d-4fc1-b2e0-60c91dad6805 with Priority: HIGHEST for action port
[  610.389650] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d51d8d24-4b4d-4fc1-b2e0-60c91dad6805 with Priority: HIGHEST for action port
[  610.390185] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.390309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.390419] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.390572] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.390659] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.390807] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-d51d8d24-4b4d-4fc1-b2e0-60c91dad6805 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.390897] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-d51d8d24-4b4d-4fc1-b2e0-60c91dad6805" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.390956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-d51d8d24-4b4d-4fc1-b2e0-60c91dad6805" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.391164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-d51d8d24-4b4d-4fc1-b2e0-60c91dad6805" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.391281] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d51d8d24-4b4d-4fc1-b2e0-60c91dad6805',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  610.391485] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 927550474a51429787c5cd145aab6fd6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  610.396528] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 927550474a51429787c5cd145aab6fd6
[  610.512224] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.512444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.512580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.512917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.513828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.513828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.513828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.513828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.513828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.514070] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.515028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.515028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.515028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.515028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.515028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.515303] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.515303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.515303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.516124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.516124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.516124] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.516124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.516309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.516387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.516492] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  610.517569] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a42710a009ab43399a88b44d2e8779b9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  610.521732] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a42710a009ab43399a88b44d2e8779b9
[  610.857801] env[60890]: INFO networking_nsxv3.common.synchronization [None req-046a7ec8-9347-4d8e-8890-4d9d0d5ba7e7 tempest-ServersTestManualDisk-2117565693 tempest-ServersTestManualDisk-2117565693-project-member] Enqueued Resource ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 with Priority: HIGHEST for action security_group_rules
[  610.857801] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 with Priority: HIGHEST for action security_group_rules
[  610.857801] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.857801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.857801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.857801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.857801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.857801] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-652ef4e3-97d8-44ab-8738-5ae16971a428 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.857801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.857801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.857801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.857801] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('652ef4e3-97d8-44ab-8738-5ae16971a428',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  610.857801] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 354247b8bd5849b7be2630fdbc5db5df in queue reply_156db6c4256a4c58884afce15e6f5340
[  610.863683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  610.863683] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  610.863683] env[60890]: ERROR oslo_messaging.rpc.server 
[  610.871786] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 354247b8bd5849b7be2630fdbc5db5df
[  610.893067] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-d51d8d24-4b4d-4fc1-b2e0-60c91dad6805" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.893393] env[60890]: Traceback (most recent call last):
[  610.893495] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  610.893495] env[60890]:     timer()
[  610.893574] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  610.893574] env[60890]:     cb(*args, **kw)
[  610.893658] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  610.893658] env[60890]:     result = function(*args, **kwargs)
[  610.893733] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  610.893733] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  610.893803] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  610.893803] env[60890]:     return method(*args, **kwargs)
[  610.893872] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  610.893872] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  610.893984] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  610.893984] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  610.894056] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  610.894056] env[60890]:     result = self.transport._send(
[  610.894127] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  610.894127] env[60890]:     return self._driver.send(target, ctxt, message,
[  610.894195] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  610.894195] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  610.894265] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  610.894265] env[60890]:     raise result
[  610.894352] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  610.894352] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  610.940716] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.940716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.940716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.940716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.940716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.940716] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.940716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.940716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.940716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.940716] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.942534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.942534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.942534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.942534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.942534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.942534] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.942534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.942534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.943135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.943388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  610.943807] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  610.944148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  610.944372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  610.944846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  610.945152] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  610.945478] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e71f45a2076149639b329e824c4df86e in queue reply_156db6c4256a4c58884afce15e6f5340
[  610.951438] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e71f45a2076149639b329e824c4df86e
[  611.018822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  611.019485] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  611.019485] env[60890]: ERROR oslo_messaging.rpc.server 
[  611.068522] env[60890]: INFO networking_nsxv3.common.synchronization [None req-bc3277e3-17f6-4500-91fe-ae1e0959b37d tempest-ServersTestManualDisk-2117565693 tempest-ServersTestManualDisk-2117565693-project-member] Enqueued Resource ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 with Priority: HIGHEST for action security_group_rules
[  611.069311] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 with Priority: HIGHEST for action security_group_rules
[  611.069390] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.069494] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.069558] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.069754] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.069833] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.069981] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-652ef4e3-97d8-44ab-8738-5ae16971a428 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.070059] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.378532] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.378734] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.378771] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.379022] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.379121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.379219] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 already deleted.
[  611.379301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.380288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.380288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.380288] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('652ef4e3-97d8-44ab-8738-5ae16971a428',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  611.380288] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 61178b16176243628075c8c21375ce2e in queue reply_156db6c4256a4c58884afce15e6f5340
[  611.382403] env[60890]: INFO networking_nsxv3.common.synchronization [req-fd7e51fd-3cea-4bc3-85fc-84fe8946b6b3 req-16843a4a-55e9-49bf-97fe-dd5ace4e29eb tempest-SecurityGroupRulesTestJSON-432430954 tempest-SecurityGroupRulesTestJSON-432430954-project-member] Enqueued Resource ID: a8165742-514c-424f-aeed-5bf70ad8974d with Priority: HIGHEST for action security_group_rules
[  611.383046] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a8165742-514c-424f-aeed-5bf70ad8974d with Priority: HIGHEST for action security_group_rules
[  611.383416] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.383529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.383860] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.384062] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.384182] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.384355] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-a8165742-514c-424f-aeed-5bf70ad8974d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.384458] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.384519] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.384722] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.384774] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('a8165742-514c-424f-aeed-5bf70ad8974d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  611.385218] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c42403429edb4e8d933df1ebb7924a76 in queue reply_156db6c4256a4c58884afce15e6f5340
[  611.400473] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c42403429edb4e8d933df1ebb7924a76
[  611.403691] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 61178b16176243628075c8c21375ce2e
[  611.451888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  611.451888] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  611.451888] env[60890]: ERROR oslo_messaging.rpc.server 
[  611.616345] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.616345] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.616345] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.616345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.616345] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.619098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.619098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.619098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.619098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.619098] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.619098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.619098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.619098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.620054] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  611.620436] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ab68155f59e24f5082596c69fcb4f49b in queue reply_156db6c4256a4c58884afce15e6f5340
[  611.630400] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ab68155f59e24f5082596c69fcb4f49b
[  611.739666] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.739883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.740023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.740327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.740477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.740734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-411 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.740893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.741010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.741223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.741465] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.741619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.741738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.741953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.742095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.742238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-411" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.742514] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.742760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.742907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.743189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.743404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.743750] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bb4e4a76-430a-43b9-be76-d0d5892c4663 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.743989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.745400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.745400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.745400] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bb4e4a76-430a-43b9-be76-d0d5892c4663',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  611.745551] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7daf16b719b74a619959dd8be12a3ebc in queue reply_156db6c4256a4c58884afce15e6f5340
[  611.757496] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.757594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.757751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.758193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.758592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.758949] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.759060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.759232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.759489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.759802] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.760028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.760186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.760441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.760612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.760794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.761078] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.761272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.761424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.761707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.761881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.762155] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.762366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.762518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.762764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.762967] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  611.763239] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 471c04b77d9b4f01a5da3fe444272baa in queue reply_156db6c4256a4c58884afce15e6f5340
[  611.764746] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7daf16b719b74a619959dd8be12a3ebc
[  611.773423] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 471c04b77d9b4f01a5da3fe444272baa
[  611.903023] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.903309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.903309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.903442] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.903536] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.903629] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 already deleted.
[  611.903714] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.907274] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  611.907480] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  611.907582] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  611.907795] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  611.908298] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  611.908576] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: a8165742-514c-424f-aeed-5bf70ad8974d already deleted.
[  611.908576] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-a8165742-514c-424f-aeed-5bf70ad8974d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.125204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  612.125734] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  612.125734] env[60890]: ERROR oslo_messaging.rpc.server 
[  612.253240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bb4e4a76-430a-43b9-be76-d0d5892c4663" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  612.253760] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  612.253760] env[60890]: ERROR oslo_messaging.rpc.server 
[  612.264766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  612.265234] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  612.265234] env[60890]: ERROR oslo_messaging.rpc.server 
[  612.490640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.492034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.492034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.492034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.492034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.492034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.492034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.492034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.492429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.492429] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.492505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.492559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.492784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.492911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.493056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.493619] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.493794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.493918] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.494128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.494263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.494499] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.494650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.494770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.495089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.495226] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  612.495459] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4ab2048529494782a2e92894762eb66f in queue reply_156db6c4256a4c58884afce15e6f5340
[  612.501640] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4ab2048529494782a2e92894762eb66f
[  612.799909] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.800155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.800284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.800571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.800719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.800985] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.801187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.801274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.801473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.801704] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.801858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.801977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.802181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.802328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.803047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.803315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.803482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.803608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.803823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.803961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.804218] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.804372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.804490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.804707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.804860] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  612.805100] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 466cb68bbdf54b87aaf3830f2a689f00 in queue reply_156db6c4256a4c58884afce15e6f5340
[  612.814236] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 466cb68bbdf54b87aaf3830f2a689f00
[  612.977337] env[60890]: INFO networking_nsxv3.common.synchronization [None req-47def653-c70b-4805-8d39-e825591bdeb9 None None] Enqueued Resource ID: 0f790c6d-70c7-4684-8504-14c9b31f038c with Priority: HIGHEST for action port
[  612.977874] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0f790c6d-70c7-4684-8504-14c9b31f038c with Priority: HIGHEST for action port
[  612.978194] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.978317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.978381] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.978580] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.978665] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.978815] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-0f790c6d-70c7-4684-8504-14c9b31f038c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  612.978912] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-0f790c6d-70c7-4684-8504-14c9b31f038c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  612.978976] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-0f790c6d-70c7-4684-8504-14c9b31f038c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  612.979179] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-0f790c6d-70c7-4684-8504-14c9b31f038c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  612.979289] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f790c6d-70c7-4684-8504-14c9b31f038c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  612.979432] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 867fbd8a62f54d71b18332b19cf43a6d in queue reply_156db6c4256a4c58884afce15e6f5340
[  612.984584] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 867fbd8a62f54d71b18332b19cf43a6d
[  612.997969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  612.998453] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  612.998453] env[60890]: ERROR oslo_messaging.rpc.server 
[  613.307098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  613.307618] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  613.307618] env[60890]: ERROR oslo_messaging.rpc.server 
[  613.481904] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-0f790c6d-70c7-4684-8504-14c9b31f038c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  613.482152] env[60890]: Traceback (most recent call last):
[  613.482192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  613.482192] env[60890]:     timer()
[  613.482192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  613.482192] env[60890]:     cb(*args, **kw)
[  613.482192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  613.482192] env[60890]:     result = function(*args, **kwargs)
[  613.482192] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  613.482192] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  613.482192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  613.482192] env[60890]:     return method(*args, **kwargs)
[  613.482192] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  613.482192] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  613.482192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  613.482192] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  613.482796] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  613.482796] env[60890]:     result = self.transport._send(
[  613.482796] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  613.482796] env[60890]:     return self._driver.send(target, ctxt, message,
[  613.482796] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  613.482796] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  613.482796] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  613.482796] env[60890]:     raise result
[  613.482796] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  613.482796] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  613.862689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  613.862689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  613.862689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  613.862689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  613.862689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  613.867750] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  613.867930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  613.868070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  613.868303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  613.868442] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  613.868683] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 45d34b780bb7447f952e4e0d12d1cc09 in queue reply_156db6c4256a4c58884afce15e6f5340
[  613.874373] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 45d34b780bb7447f952e4e0d12d1cc09
[  614.162211] env[60890]: INFO networking_nsxv3.common.synchronization [req-85a55bd4-b87c-4070-b69e-36a38395bf69 req-94535d81-c959-43db-afe7-7a4e5d46d7d9 tempest-ServerRescueNegativeTestJSON-1640230806 tempest-ServerRescueNegativeTestJSON-1640230806-project-member] Enqueued Resource ID: e314c230-88be-4118-8948-b5f480080893 with Priority: HIGHEST for action security_group_members
[  614.163322] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e314c230-88be-4118-8948-b5f480080893 with Priority: HIGHEST for action security_group_members
[  614.163322] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.163322] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.163322] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.163546] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.163581] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.163719] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e314c230-88be-4118-8948-b5f480080893 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.163816] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.163876] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.164044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.164189] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.164274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.164331] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.164468] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.164549] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.164636] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e314c230-88be-4118-8948-b5f480080893',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  614.164852] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4def2e071136402da98f4687d7a02590 in queue reply_156db6c4256a4c58884afce15e6f5340
[  614.172913] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4def2e071136402da98f4687d7a02590
[  614.194751] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.194973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.195100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.195357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.195887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.195887] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.196017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.196111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.196481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.196611] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.196710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.196822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.197025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.197161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.197327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.197549] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.197708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.197997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.198065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.198156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.198389] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.198578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.198650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.198853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.198986] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  614.199210] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b1868b6bfb3a4d10874930963d0d1727 in queue reply_156db6c4256a4c58884afce15e6f5340
[  614.205597] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b1868b6bfb3a4d10874930963d0d1727
[  614.372493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  614.373036] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  614.373036] env[60890]: ERROR oslo_messaging.rpc.server 
[  614.667118] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.667302] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.667450] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.667591] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.667689] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.667886] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.667981] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.668068] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.668205] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.668282] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.668370] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e314c230-88be-4118-8948-b5f480080893 already deleted.
[  614.668451] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e314c230-88be-4118-8948-b5f480080893" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.701737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  614.702340] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  614.702340] env[60890]: ERROR oslo_messaging.rpc.server 
[  614.829292] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.829490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.829619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.829884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.830035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.830296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.830433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.830546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.830771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.830997] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.831276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.831276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.831457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.831594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.831739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.831980] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.832171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.832288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.832513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.832645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  614.833125] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  614.833125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  614.833253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  614.833381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  614.833514] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  614.833738] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f1c683bb34f6423c96f6a83d05fc0d25 in queue reply_156db6c4256a4c58884afce15e6f5340
[  614.868048] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f1c683bb34f6423c96f6a83d05fc0d25
[  615.093510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  615.093722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  615.093854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  615.094123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  615.094272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.094531] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  615.094753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  615.094823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  615.095047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  615.095294] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  615.095455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  615.095562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  615.095801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  615.095957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.096126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.096380] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  615.096534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  615.096653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  615.096917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  615.097065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.097306] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  615.097459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  615.097579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  615.098088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  615.098088] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  615.098198] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 25c6f875ccd44951b13e134920076515 in queue reply_156db6c4256a4c58884afce15e6f5340
[  615.150769] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 25c6f875ccd44951b13e134920076515
[  615.336110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  615.336622] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  615.336622] env[60890]: ERROR oslo_messaging.rpc.server 
[  615.437807] env[60890]: INFO networking_nsxv3.common.synchronization [None req-d6e41f4c-dfcc-4030-b2fb-99d74d762c0a None None] Enqueued Resource ID: 9047384d-a418-47e6-bde6-32f679a0aaef with Priority: HIGHEST for action port
[  615.437926] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9047384d-a418-47e6-bde6-32f679a0aaef with Priority: HIGHEST for action port
[  615.438524] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  615.438646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  615.438720] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  615.438918] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  615.439160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.439160] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-9047384d-a418-47e6-bde6-32f679a0aaef taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  615.439268] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-9047384d-a418-47e6-bde6-32f679a0aaef" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  615.439308] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-9047384d-a418-47e6-bde6-32f679a0aaef" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  615.439480] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-9047384d-a418-47e6-bde6-32f679a0aaef" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  615.439555] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9047384d-a418-47e6-bde6-32f679a0aaef',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  615.439727] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 69737c883139418db610f1e0bdc2f373 in queue reply_156db6c4256a4c58884afce15e6f5340
[  615.456516] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 69737c883139418db610f1e0bdc2f373
[  615.602755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  615.603278] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  615.603278] env[60890]: ERROR oslo_messaging.rpc.server 
[  615.942032] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-9047384d-a418-47e6-bde6-32f679a0aaef" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  615.942275] env[60890]: Traceback (most recent call last):
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  615.942343] env[60890]:     timer()
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  615.942343] env[60890]:     cb(*args, **kw)
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  615.942343] env[60890]:     result = function(*args, **kwargs)
[  615.942343] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  615.942343] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  615.942343] env[60890]:     return method(*args, **kwargs)
[  615.942343] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  615.942343] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  615.942343] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  615.942343] env[60890]:     result = self.transport._send(
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  615.942343] env[60890]:     return self._driver.send(target, ctxt, message,
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  615.942343] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  615.942343] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  615.942343] env[60890]:     raise result
[  615.942343] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  615.942343] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  616.020098] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.020297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.020528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.020690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.020862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.021136] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.021289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.021409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.021626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.021871] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.022029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.022147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.022355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.022495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.022642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.022893] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.023049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.023166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.023371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.023504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.023740] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.023901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.024071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.024251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.024384] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bdf0b44c-58fe-4ecb-b63d-cb5382f772e9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  616.024616] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1efe5b598bd94e078b966a85d9501556 in queue reply_156db6c4256a4c58884afce15e6f5340
[  616.034925] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1efe5b598bd94e078b966a85d9501556
[  616.351403] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.351691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.351730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.351991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.352180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.352472] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.352635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.352985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.352985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.353213] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.353369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.353486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.353722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.353874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.354070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.354336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.354491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.354609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.354855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.355018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.355269] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  616.355427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  616.355545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  616.355793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  616.355932] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  616.357705] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 46b5036a4b84473ab000b0ae35a1fe68 in queue reply_156db6c4256a4c58884afce15e6f5340
[  616.368560] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 46b5036a4b84473ab000b0ae35a1fe68
[  616.526531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bdf0b44c-58fe-4ecb-b63d-cb5382f772e9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  616.527069] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  616.527069] env[60890]: ERROR oslo_messaging.rpc.server 
[  616.859299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  616.859816] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  616.859816] env[60890]: ERROR oslo_messaging.rpc.server 
[  617.646281] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.646582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.646619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.646895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.647048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.647303] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.647461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.647581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.647799] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.648056] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.648215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.648332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.648541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.648683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.648842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.649107] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.649260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.649381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.649589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.649726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.649970] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.653683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.653822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.654295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.654295] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  617.654432] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 571ffad5dde14b209732869b8f9b45bb in queue reply_156db6c4256a4c58884afce15e6f5340
[  617.660175] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 571ffad5dde14b209732869b8f9b45bb
[  617.766222] env[60890]: INFO networking_nsxv3.common.synchronization [req-1b1d843e-1b51-4030-b1d1-4b2e0d4e78c0 req-14e730b1-3e29-4d8e-b99d-840be8c61254 tempest-ListServersNegativeTestJSON-955562437 tempest-ListServersNegativeTestJSON-955562437-project-member] Enqueued Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  617.766759] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  617.767080] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.767199] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.767263] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.767455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.767539] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.767694] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-976f3f29-8ecd-43ae-ac0b-7b7127db7112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.767794] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.767850] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.768012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.768178] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.768265] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.768325] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.768466] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.768546] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.768633] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('976f3f29-8ecd-43ae-ac0b-7b7127db7112',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  617.768823] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 52ae3d2ce2464de6a6a665f72b4bd391 in queue reply_156db6c4256a4c58884afce15e6f5340
[  617.771679] env[60890]: INFO networking_nsxv3.common.synchronization [req-d6583788-6091-445e-b904-99edd86ac735 req-90249d59-6774-4c6c-92ef-a11d56989620 tempest-DeleteServersAdminTestJSON-570518421 tempest-DeleteServersAdminTestJSON-570518421-project-member] Enqueued Resource ID: 86656952-d254-40e4-a34d-b4f64574ede0 with Priority: HIGHEST for action security_group_members
[  617.772110] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 86656952-d254-40e4-a34d-b4f64574ede0 with Priority: HIGHEST for action security_group_members
[  617.772382] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.772533] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.772576] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.772725] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.772807] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.772950] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-86656952-d254-40e4-a34d-b4f64574ede0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.773042] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.773099] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.773240] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.773373] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  617.773464] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  617.773528] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  617.773656] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  617.773734] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  617.773817] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('86656952-d254-40e4-a34d-b4f64574ede0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  617.773966] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 03d0cd5a52214c9dbf43cbf1573b22fc in queue reply_156db6c4256a4c58884afce15e6f5340
[  617.782717] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 52ae3d2ce2464de6a6a665f72b4bd391
[  617.782717] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 03d0cd5a52214c9dbf43cbf1573b22fc
[  618.157099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  618.157630] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  618.157630] env[60890]: ERROR oslo_messaging.rpc.server 
[  618.270892] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.271081] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.271132] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.271359] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.271455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.271994] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.271994] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.271994] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.271994] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.272310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.272310] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 already deleted.
[  618.272310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.275759] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.275969] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.276057] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.276231] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.276354] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.276545] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.276646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.276708] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.276861] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.276938] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.277024] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 86656952-d254-40e4-a34d-b4f64574ede0 already deleted.
[  618.277100] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-86656952-d254-40e4-a34d-b4f64574ede0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.836841] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.837150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.837452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.837452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.837592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.837859] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.837977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.838097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.838317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.838564] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.838723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.838846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.839058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.839202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.839352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.839603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.839760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.839882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.840120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.840260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  618.840501] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  618.840657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  618.840778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  618.840991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  618.841127] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  618.841360] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ed762e153b9f49a695e2de15bef703be in queue reply_156db6c4256a4c58884afce15e6f5340
[  618.846777] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ed762e153b9f49a695e2de15bef703be
[  619.343926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  619.344469] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  619.344469] env[60890]: ERROR oslo_messaging.rpc.server 
[  619.421668] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.421875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.421999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.422270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.422413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.422759] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.423147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.423147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.423277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.423489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.423639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.423754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.423966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.424122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.424272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.424519] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.424671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.424851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.425061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.425194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.425438] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.425586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.425717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.426012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.426144] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  619.426376] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg df08285174c3411584fdc32e5bd673cd in queue reply_156db6c4256a4c58884afce15e6f5340
[  619.435006] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg df08285174c3411584fdc32e5bd673cd
[  619.628408] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.628604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.628737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.629011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.629159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.629398] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.629551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.629686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.629967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.630245] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.630450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.630594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.630815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.630954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.631101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.631376] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.631544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.631676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.631885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.632351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.632678] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  619.632914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  619.633090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  619.633764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  619.633978] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  619.634272] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 26dac04a565b4274ab38b14903db8142 in queue reply_156db6c4256a4c58884afce15e6f5340
[  619.660177] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 26dac04a565b4274ab38b14903db8142
[  619.932471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  619.932849] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  619.932849] env[60890]: ERROR oslo_messaging.rpc.server 
[  620.136644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  620.137189] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  620.137189] env[60890]: ERROR oslo_messaging.rpc.server 
[  620.300479] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.300692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.300822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.301094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.301242] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.301498] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.301650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.301765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.301972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.302213] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.302371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.302487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.302721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.302866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.303015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.303260] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.303410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.303527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.303769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.306148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.306428] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.306673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.306820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.307037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.307171] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  620.307399] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 607ca250a99e47bea39de4603c407e13 in queue reply_156db6c4256a4c58884afce15e6f5340
[  620.312695] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 607ca250a99e47bea39de4603c407e13
[  620.809358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  620.809873] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  620.809873] env[60890]: ERROR oslo_messaging.rpc.server 
[  620.817268] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.817470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.817601] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.817897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.818051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.818305] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.818467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.818579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.818801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.819049] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.819205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.819322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.819529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.819672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.819841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.820112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.820304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.820389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.820597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.820735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  620.821282] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-23770fb7-61c9-48cd-84d4-20de37df3b75 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  620.821461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  620.821584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  620.821805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  620.821947] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('23770fb7-61c9-48cd-84d4-20de37df3b75',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  620.822177] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2fbfa56cfd5f49c1bf5779c6d49634d7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  620.827792] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2fbfa56cfd5f49c1bf5779c6d49634d7
[  621.324457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-23770fb7-61c9-48cd-84d4-20de37df3b75" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  621.325227] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  621.325227] env[60890]: ERROR oslo_messaging.rpc.server 
[  621.390391] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  621.390605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  621.390753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  621.390999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  621.391145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  621.391399] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  621.391556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  621.395895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  621.395895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  621.395895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  621.395895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  621.395895] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  621.395895] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d0d9fb7bf4a248fb87e1fdf3557393bb in queue reply_156db6c4256a4c58884afce15e6f5340
[  621.403100] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d0d9fb7bf4a248fb87e1fdf3557393bb
[  621.897158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  621.897688] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  621.897688] env[60890]: ERROR oslo_messaging.rpc.server 
[  622.523126] env[60890]: INFO networking_nsxv3.common.synchronization [req-1b1d843e-1b51-4030-b1d1-4b2e0d4e78c0 req-9413fdee-2552-4aa6-915b-0f62b6681932 tempest-ListServersNegativeTestJSON-955562437 tempest-ListServersNegativeTestJSON-955562437-project-member] Enqueued Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  622.523655] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  622.523975] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.524138] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.524206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.524414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.524501] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  622.524657] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-976f3f29-8ecd-43ae-ac0b-7b7127db7112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.524816] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.524848] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.524985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.525126] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.525239] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.525273] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.525479] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.525572] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  622.525667] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('976f3f29-8ecd-43ae-ac0b-7b7127db7112',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  622.525845] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3e13ec7f67d245a8aefe86c0abee861f in queue reply_156db6c4256a4c58884afce15e6f5340
[  622.537484] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3e13ec7f67d245a8aefe86c0abee861f
[  622.548712] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.548977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.549148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.549464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.549640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  622.549954] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.550151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.550308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.550819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.551547] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.551547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.551547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.551762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.553208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  622.553424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  622.553733] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.553922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.554072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.554324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.555065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  622.555616] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  622.556635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  622.556789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  622.557523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  622.557523] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  622.557523] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 12ec8764409b476f88c6d41cbe0ddfb2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  622.568038] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 12ec8764409b476f88c6d41cbe0ddfb2
[  623.028187] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  623.028345] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  623.028415] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  623.028634] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  623.028734] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  623.028933] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  623.029035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  623.029098] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  623.029251] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  623.029326] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  623.029407] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 already deleted.
[  623.029488] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  623.060385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  623.060901] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  623.060901] env[60890]: ERROR oslo_messaging.rpc.server 
[  624.232711] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  624.233036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  624.233036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  624.233411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  624.233446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  624.233746] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  624.233972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  624.234106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  624.234338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  624.234589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  624.234797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  624.234928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  624.235142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  624.235294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  624.235446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  624.235699] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  624.236210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  624.236343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  624.236553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  624.236729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  624.236963] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  624.237162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  624.237280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  624.237488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  624.237621] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  624.237858] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 01ff1cf7dafd4d0bb7ae754236fdc9f5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  624.253727] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 01ff1cf7dafd4d0bb7ae754236fdc9f5
[  624.448821] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c2076f1ac33746518273302396074f16 in queue reply_156db6c4256a4c58884afce15e6f5340
[  624.471296] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c2076f1ac33746518273302396074f16
[  624.739615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  624.740153] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  624.740153] env[60890]: ERROR oslo_messaging.rpc.server 
[  624.985614] env[60890]: INFO networking_nsxv3.common.synchronization [None req-22a4ff2b-d6c1-4acb-a705-f6b336b2d43f None None] Enqueued Resource ID: 28ccdcf7-cad6-4916-83a0-b4e996d4d6a1 with Priority: HIGHEST for action port
[  624.985874] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 28ccdcf7-cad6-4916-83a0-b4e996d4d6a1 with Priority: HIGHEST for action port
[  624.986812] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  624.986948] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  624.987032] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  624.987240] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  624.987325] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  624.987475] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-28ccdcf7-cad6-4916-83a0-b4e996d4d6a1 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  624.987569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-28ccdcf7-cad6-4916-83a0-b4e996d4d6a1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  624.987627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-28ccdcf7-cad6-4916-83a0-b4e996d4d6a1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  624.987820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-28ccdcf7-cad6-4916-83a0-b4e996d4d6a1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  624.987914] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('28ccdcf7-cad6-4916-83a0-b4e996d4d6a1',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  624.988102] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 53d4ad492c0f4f929c13bfc50f28d872 in queue reply_156db6c4256a4c58884afce15e6f5340
[  624.994793] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 53d4ad492c0f4f929c13bfc50f28d872
[  625.404477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  625.404477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  625.404477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  625.404477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  625.404477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  625.404477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  625.406368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  625.406368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  625.406368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  625.406368] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  625.406368] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6b45cd33f5cb4dd9a3677b7d0c673a3c in queue reply_156db6c4256a4c58884afce15e6f5340
[  625.439150] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6b45cd33f5cb4dd9a3677b7d0c673a3c
[  625.490295] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-28ccdcf7-cad6-4916-83a0-b4e996d4d6a1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  625.490550] env[60890]: Traceback (most recent call last):
[  625.490590] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  625.490590] env[60890]:     timer()
[  625.490590] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  625.490590] env[60890]:     cb(*args, **kw)
[  625.490590] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  625.490590] env[60890]:     result = function(*args, **kwargs)
[  625.490590] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  625.490590] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  625.490590] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  625.490590] env[60890]:     return method(*args, **kwargs)
[  625.490590] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  625.490590] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  625.490590] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  625.490590] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  625.490590] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  625.490590] env[60890]:     result = self.transport._send(
[  625.491023] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  625.491023] env[60890]:     return self._driver.send(target, ctxt, message,
[  625.491023] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  625.491023] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  625.491023] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  625.491023] env[60890]:     raise result
[  625.491023] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  625.491023] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  625.907715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  625.908309] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  625.908309] env[60890]: ERROR oslo_messaging.rpc.server 
[  626.525205] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  626.525481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  626.525558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  626.525818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  626.525957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  626.526211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  626.526427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  626.526578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  626.526803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  626.527068] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  626.527254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  626.527346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  626.527558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  626.527704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  626.527859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  626.534200] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  626.534270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  626.534345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  626.540289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  626.540510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  626.540843] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  626.541024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  626.541154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  626.541410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  626.541625] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  626.541919] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 33f88f2e215f4639b392d823ab241e26 in queue reply_156db6c4256a4c58884afce15e6f5340
[  626.547503] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 33f88f2e215f4639b392d823ab241e26
[  627.045369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  627.045998] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  627.045998] env[60890]: ERROR oslo_messaging.rpc.server 
[  627.643905] env[60890]: INFO networking_nsxv3.common.synchronization [None req-c64b522b-4e31-490e-9be3-ef895f0e7150 None None] Enqueued Resource ID: ac9bc63e-4e7a-4248-88fb-0be86e306b3d with Priority: HIGHEST for action port
[  627.644322] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ac9bc63e-4e7a-4248-88fb-0be86e306b3d with Priority: HIGHEST for action port
[  627.645153] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  627.645278] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  627.645345] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  627.645491] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  627.645577] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  627.645858] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-ac9bc63e-4e7a-4248-88fb-0be86e306b3d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  627.645982] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-ac9bc63e-4e7a-4248-88fb-0be86e306b3d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  627.646052] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-ac9bc63e-4e7a-4248-88fb-0be86e306b3d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  627.646337] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-ac9bc63e-4e7a-4248-88fb-0be86e306b3d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  627.646427] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ac9bc63e-4e7a-4248-88fb-0be86e306b3d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  627.646617] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ccfc70b44dd445d18383854a3e6e065b in queue reply_156db6c4256a4c58884afce15e6f5340
[  627.655509] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ccfc70b44dd445d18383854a3e6e065b
[  627.689981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  627.690159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  627.690287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  627.690548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  627.690697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  627.690950] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  627.691099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  627.691217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  627.691496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  627.691680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  627.691838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  627.691958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  627.692206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  627.692345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  627.692493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  627.692979] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  627.693159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  627.693288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  627.693518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  627.693702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  627.693972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  627.694133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  627.694256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  627.694495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  627.694637] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  627.694968] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fa71b2ab14d84eef92186520ebba2252 in queue reply_156db6c4256a4c58884afce15e6f5340
[  627.702435] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fa71b2ab14d84eef92186520ebba2252
[  628.148429] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-ac9bc63e-4e7a-4248-88fb-0be86e306b3d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  628.148678] env[60890]: Traceback (most recent call last):
[  628.148719] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  628.148719] env[60890]:     timer()
[  628.148719] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  628.148719] env[60890]:     cb(*args, **kw)
[  628.148719] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  628.148719] env[60890]:     result = function(*args, **kwargs)
[  628.148719] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  628.148719] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  628.148719] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  628.148719] env[60890]:     return method(*args, **kwargs)
[  628.148719] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  628.148719] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  628.148719] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  628.148719] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  628.148719] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  628.148719] env[60890]:     result = self.transport._send(
[  628.148719] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  628.148719] env[60890]:     return self._driver.send(target, ctxt, message,
[  628.148719] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  628.148719] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  628.149222] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  628.149222] env[60890]:     raise result
[  628.149222] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  628.149222] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  628.197459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  628.198001] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  628.198001] env[60890]: ERROR oslo_messaging.rpc.server 
[  628.370877] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  628.371086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  628.371217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  628.371496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  628.371648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  628.371909] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  628.372102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  628.372231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  628.372447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  628.372710] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  628.372886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  628.373033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  628.373254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  628.373401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  628.373553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  628.373805] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  628.373959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  628.374089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  628.374299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  628.374436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  628.374718] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  628.374847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  628.374971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  628.375183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  628.375319] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  628.375557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 661e744effa04a3898fabfe667c1106e in queue reply_156db6c4256a4c58884afce15e6f5340
[  628.382866] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 661e744effa04a3898fabfe667c1106e
[  628.877846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  628.878418] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  628.878418] env[60890]: ERROR oslo_messaging.rpc.server 
[  629.506568] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  629.506568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  629.506568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  629.506568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  629.506568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  629.506885] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  629.506951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  629.507089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  629.507308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  629.507558] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  629.507718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  629.507853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  629.508079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  629.508227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  629.508377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  629.508633] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  629.508785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  629.508903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  629.509108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  629.509240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  629.509477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  629.509628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  629.509745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  629.509948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  629.510083] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  629.510310] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cbaed5e4db9e45b3ac41cb807ef62b9b in queue reply_156db6c4256a4c58884afce15e6f5340
[  629.515871] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cbaed5e4db9e45b3ac41cb807ef62b9b
[  630.013291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4b282eb0-8a7c-47f2-87fc-c2bdd35fc0d4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  630.013823] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  630.013823] env[60890]: ERROR oslo_messaging.rpc.server 
[  631.192385] env[60890]: INFO networking_nsxv3.common.synchronization [req-1b1d843e-1b51-4030-b1d1-4b2e0d4e78c0 req-297a8c2b-83c7-434e-b378-56d940b9f6c9 tempest-ListServersNegativeTestJSON-955562437 tempest-ListServersNegativeTestJSON-955562437-project-member] Enqueued Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  631.192950] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  631.193279] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  631.193408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  631.193477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  631.193673] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  631.193758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  631.193914] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-976f3f29-8ecd-43ae-ac0b-7b7127db7112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  631.194007] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  631.194066] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  631.194210] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  631.194349] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  631.194438] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  631.194499] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  631.194635] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  631.194772] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  631.194861] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('976f3f29-8ecd-43ae-ac0b-7b7127db7112',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  631.195034] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3964749a89f64ebabda8dc31b21485d6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  631.203310] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3964749a89f64ebabda8dc31b21485d6
[  631.698549] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  631.698706] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  631.698769] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  631.698995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  631.699093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  631.699282] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  631.699386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  631.699450] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  631.699605] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  631.699685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  631.699773] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 already deleted.
[  631.699847] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.056795] env[60890]: INFO networking_nsxv3.common.synchronization [req-1b1d843e-1b51-4030-b1d1-4b2e0d4e78c0 req-d58a2406-ce5b-442a-be77-fb5e346068a6 tempest-ListServersNegativeTestJSON-955562437 tempest-ListServersNegativeTestJSON-955562437-project-member] Enqueued Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  635.057330] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  635.057649] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.057771] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.057840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.058034] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.058116] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.058270] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-976f3f29-8ecd-43ae-ac0b-7b7127db7112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.058366] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.058427] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.058578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.058713] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.058803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.058862] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.058993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.059072] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.059158] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('976f3f29-8ecd-43ae-ac0b-7b7127db7112',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  635.059329] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg bb7ecff1eb6649c9b342ec6a0408751d in queue reply_156db6c4256a4c58884afce15e6f5340
[  635.073383] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bb7ecff1eb6649c9b342ec6a0408751d
[  635.565001] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.565164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.565236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.565454] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.565916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.565916] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.565916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.566071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.566181] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.566260] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.566346] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 already deleted.
[  635.566432] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.584308] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.584494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.584623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.584921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.585074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.585330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.585489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.585607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.585864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.586123] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.586283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.586463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.586705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.586857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.587008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.587265] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.587441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.587536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.587854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.587929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  635.588184] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  635.588345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  635.588495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  635.588733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  635.588872] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  635.589099] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 251465b35f8f4f2aa3f1a3fe80fd79c9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  635.596207] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 251465b35f8f4f2aa3f1a3fe80fd79c9
[  636.091799] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  636.092387] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  636.092387] env[60890]: ERROR oslo_messaging.rpc.server 
[  636.690945] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  636.691149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  636.691278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  636.691550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  636.691779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  636.692095] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  636.692263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  636.692401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  636.692604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  636.692852] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  636.693009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  636.693134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  636.693382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  636.693578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  636.693742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  636.693995] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  636.694152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  636.694270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  636.694489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  636.694632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  636.694926] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  636.695110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  636.695234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  636.695445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  636.695580] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  636.695809] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 60efb7d6f86f4d9c9de24488926f4905 in queue reply_156db6c4256a4c58884afce15e6f5340
[  636.701544] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 60efb7d6f86f4d9c9de24488926f4905
[  637.198123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  637.198659] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  637.198659] env[60890]: ERROR oslo_messaging.rpc.server 
[  637.300073] env[60890]: INFO networking_nsxv3.common.synchronization [None req-40300bf0-8482-4e8a-947b-1ff734a957ed None None] Enqueued Resource ID: 91de27c8-208c-465e-9ad7-e684020f0163 with Priority: HIGHEST for action port
[  637.300374] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 91de27c8-208c-465e-9ad7-e684020f0163 with Priority: HIGHEST for action port
[  637.300908] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.301038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.301109] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.301309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.301395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.301548] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-91de27c8-208c-465e-9ad7-e684020f0163 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.301645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-91de27c8-208c-465e-9ad7-e684020f0163" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.301710] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-91de27c8-208c-465e-9ad7-e684020f0163" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.301902] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-91de27c8-208c-465e-9ad7-e684020f0163" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.301982] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('91de27c8-208c-465e-9ad7-e684020f0163',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  637.302157] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4531937d9701480bb679653492c16eec in queue reply_156db6c4256a4c58884afce15e6f5340
[  637.310103] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4531937d9701480bb679653492c16eec
[  637.435342] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.435552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.435688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.435964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.436181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.436480] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.436644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.437675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.437675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.437675] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.437675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.437675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.437675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.438019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.438019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.438205] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.438363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.438482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.438691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.441564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.441564] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.441564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.441564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.441564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.441564] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  637.441564] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 04c35c52f15141078b41077a8daed8b1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  637.452322] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 04c35c52f15141078b41077a8daed8b1
[  637.805018] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-91de27c8-208c-465e-9ad7-e684020f0163" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.805269] env[60890]: Traceback (most recent call last):
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  637.805315] env[60890]:     timer()
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  637.805315] env[60890]:     cb(*args, **kw)
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  637.805315] env[60890]:     result = function(*args, **kwargs)
[  637.805315] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  637.805315] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  637.805315] env[60890]:     return method(*args, **kwargs)
[  637.805315] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  637.805315] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  637.805315] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  637.805315] env[60890]:     result = self.transport._send(
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  637.805315] env[60890]:     return self._driver.send(target, ctxt, message,
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  637.805315] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  637.805315] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  637.805315] env[60890]:     raise result
[  637.805892] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  637.805892] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  637.845992] env[60890]: INFO networking_nsxv3.common.synchronization [None req-20b1b955-0bc3-4bca-96fd-e7f038b144ad None None] Enqueued Resource ID: 5366d5d2-ecf9-46c8-92f9-898a87b860f1 with Priority: HIGHEST for action port
[  637.845992] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5366d5d2-ecf9-46c8-92f9-898a87b860f1 with Priority: HIGHEST for action port
[  637.846363] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.846363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.846438] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.846620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.846713] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.846859] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-5366d5d2-ecf9-46c8-92f9-898a87b860f1 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.846949] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-5366d5d2-ecf9-46c8-92f9-898a87b860f1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.847115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-5366d5d2-ecf9-46c8-92f9-898a87b860f1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.847288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-5366d5d2-ecf9-46c8-92f9-898a87b860f1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.847361] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5366d5d2-ecf9-46c8-92f9-898a87b860f1',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  637.847537] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9e89b94ddd1d4679bbbc8419c3a94b5d in queue reply_156db6c4256a4c58884afce15e6f5340
[  637.859622] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9e89b94ddd1d4679bbbc8419c3a94b5d
[  637.879194] env[60890]: INFO networking_nsxv3.common.synchronization [req-b6d8a71d-b17f-45e2-abce-d7f249060f28 req-eaf3a02c-b596-476c-9c43-a02c1bd16f55 tempest-MigrationsAdminTest-320299187 tempest-MigrationsAdminTest-320299187-project-member] Enqueued Resource ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b with Priority: HIGHEST for action security_group_members
[  637.879194] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b with Priority: HIGHEST for action security_group_members
[  637.879194] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.879194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.879194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.879194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.879194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.879194] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-18ba6c5b-4074-4f30-b957-61d36273ed7b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.879194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.879194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.879194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.879194] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  637.881402] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  637.881402] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  637.881402] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  637.881402] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.881402] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('18ba6c5b-4074-4f30-b957-61d36273ed7b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  637.881402] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6d13f642fb7542f9b2da6deaa6a5d1fe in queue reply_156db6c4256a4c58884afce15e6f5340
[  637.929884] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6d13f642fb7542f9b2da6deaa6a5d1fe
[  637.943764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  637.944669] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  637.944669] env[60890]: ERROR oslo_messaging.rpc.server 
[  638.325075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.325075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.325075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.325075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.325075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.325514] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.325514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.325514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.326650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.326650] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.326650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.326650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.326650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.326650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.326975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.326975] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.329056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.329056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.329056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.329056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.329056] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.329056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.329056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.329056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.329056] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  638.329056] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 15fcc7d97cb4462f87e1e1ede45a5c18 in queue reply_156db6c4256a4c58884afce15e6f5340
[  638.335226] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 15fcc7d97cb4462f87e1e1ede45a5c18
[  638.351497] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-5366d5d2-ecf9-46c8-92f9-898a87b860f1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.351724] env[60890]: Traceback (most recent call last):
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  638.351770] env[60890]:     timer()
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  638.351770] env[60890]:     cb(*args, **kw)
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  638.351770] env[60890]:     result = function(*args, **kwargs)
[  638.351770] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  638.351770] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  638.351770] env[60890]:     return method(*args, **kwargs)
[  638.351770] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  638.351770] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  638.351770] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  638.351770] env[60890]:     result = self.transport._send(
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  638.351770] env[60890]:     return self._driver.send(target, ctxt, message,
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  638.351770] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  638.351770] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  638.351770] env[60890]:     raise result
[  638.352468] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  638.352468] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  638.386844] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.386844] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.386844] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b already deleted.
[  638.386844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.538675] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.538675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.538675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.538957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.539109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.539400] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.539738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.539738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.539925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.540688] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.540860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.540986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.541209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.541353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.541503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.542935] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.542935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.542935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.542935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.542935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.542935] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  638.542935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  638.542935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  638.543293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  638.543293] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  638.543487] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 753772269b204d84b1fc0407aa7e239f in queue reply_156db6c4256a4c58884afce15e6f5340
[  638.548703] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 753772269b204d84b1fc0407aa7e239f
[  638.831404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  638.834489] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  638.834489] env[60890]: ERROR oslo_messaging.rpc.server 
[  639.046681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  639.047185] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  639.047185] env[60890]: ERROR oslo_messaging.rpc.server 
[  639.620296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.620296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.620296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.620296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.621002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.621002] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.621002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.621113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.621271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.621405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.621649] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.621804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.622031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.622266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.622410] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  639.622646] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f6ff94e9c9ba4e76926764c88e650604 in queue reply_156db6c4256a4c58884afce15e6f5340
[  639.739070] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.739070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.739070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.739257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.739541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.739684] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.739807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.739903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.740469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.740469] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.740610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.740648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.741743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.741743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.741743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.741743] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.742248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.742248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.742248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.742446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.742973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.742973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.742973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.743206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.743240] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  639.743483] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 90a37ebb98fb41298578c0a7e41f0a7f in queue reply_156db6c4256a4c58884afce15e6f5340
[  639.748491] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f6ff94e9c9ba4e76926764c88e650604
[  639.750300] env[60890]: INFO networking_nsxv3.common.synchronization [req-1ab9a664-6cb8-44e9-9561-473bf8192aaf req-bfebd10e-de1c-43c4-8be4-f5792204ad93 tempest-ServerExternalEventsTest-384691146 tempest-ServerExternalEventsTest-384691146-project-member] Enqueued Resource ID: 7501ba73-c2dc-466a-a7ca-3bc2146756d9 with Priority: HIGHEST for action security_group_members
[  639.750899] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7501ba73-c2dc-466a-a7ca-3bc2146756d9 with Priority: HIGHEST for action security_group_members
[  639.751454] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.751540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.751600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.751811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.751811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.751935] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7501ba73-c2dc-466a-a7ca-3bc2146756d9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.752222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7501ba73-c2dc-466a-a7ca-3bc2146756d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.752222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7501ba73-c2dc-466a-a7ca-3bc2146756d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.752343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7501ba73-c2dc-466a-a7ca-3bc2146756d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.752507] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  639.752507] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  639.752642] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  639.752934] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  639.752934] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  639.752934] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7501ba73-c2dc-466a-a7ca-3bc2146756d9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  639.753095] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ca2d97beaba5472ebb7f3e5768ba8c8e in queue reply_156db6c4256a4c58884afce15e6f5340
[  639.763014] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 90a37ebb98fb41298578c0a7e41f0a7f
[  639.810006] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ca2d97beaba5472ebb7f3e5768ba8c8e
[  640.091059] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.091673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.091673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.091673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.091867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.092344] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.092344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.092486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.093259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.093259] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.093259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.093259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.093524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.093558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.093757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.094681] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.094681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.094681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.095002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.095110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.095455] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.095621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.095745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.096073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.096230] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  640.096473] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9602dea26ccd4333a5e5a1f6596e964b in queue reply_156db6c4256a4c58884afce15e6f5340
[  640.124895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  640.125438] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  640.125438] env[60890]: ERROR oslo_messaging.rpc.server 
[  640.151906] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9602dea26ccd4333a5e5a1f6596e964b
[  640.153627] env[60890]: INFO networking_nsxv3.common.synchronization [None req-0ccfbe5c-f326-4df7-b643-a97a5466534b None None] Enqueued Resource ID: a40e8e99-5c13-4030-b11b-4a09b256c798 with Priority: HIGHEST for action port
[  640.153627] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a40e8e99-5c13-4030-b11b-4a09b256c798 with Priority: HIGHEST for action port
[  640.153627] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.153627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.153775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.153960] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.153960] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.154070] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-a40e8e99-5c13-4030-b11b-4a09b256c798 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.154162] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-a40e8e99-5c13-4030-b11b-4a09b256c798" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.154306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-a40e8e99-5c13-4030-b11b-4a09b256c798" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.154414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-a40e8e99-5c13-4030-b11b-4a09b256c798" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.154490] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a40e8e99-5c13-4030-b11b-4a09b256c798',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  640.154667] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg dc050feff964431ea8a7f38ba7cd9875 in queue reply_156db6c4256a4c58884afce15e6f5340
[  640.165926] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dc050feff964431ea8a7f38ba7cd9875
[  640.241093] env[60890]: INFO networking_nsxv3.common.synchronization [None req-58a1204c-f891-43b0-8637-9baa7cb3efea None None] Enqueued Resource ID: bd857875-8626-4dca-adc6-652fd3699b66 with Priority: HIGHEST for action port
[  640.241093] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: bd857875-8626-4dca-adc6-652fd3699b66 with Priority: HIGHEST for action port
[  640.241093] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.241093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.241093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.241448] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.241448] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.241448] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-bd857875-8626-4dca-adc6-652fd3699b66 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.241448] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-bd857875-8626-4dca-adc6-652fd3699b66" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.241601] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-bd857875-8626-4dca-adc6-652fd3699b66" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.241689] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-bd857875-8626-4dca-adc6-652fd3699b66" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.241730] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bd857875-8626-4dca-adc6-652fd3699b66',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  640.241912] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 81df1562ec7546129fd822c2cf0463c1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  640.249030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  640.249438] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  640.249438] env[60890]: ERROR oslo_messaging.rpc.server 
[  640.250591] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 81df1562ec7546129fd822c2cf0463c1
[  640.254773] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.254894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.254970] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.255134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.255225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.255404] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.255505] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.255561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.255714] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.255803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.255888] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7501ba73-c2dc-466a-a7ca-3bc2146756d9 already deleted.
[  640.255968] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7501ba73-c2dc-466a-a7ca-3bc2146756d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.598972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  640.599521] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  640.599521] env[60890]: ERROR oslo_messaging.rpc.server 
[  640.657291] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-a40e8e99-5c13-4030-b11b-4a09b256c798" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.657585] env[60890]: Traceback (most recent call last):
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  640.657585] env[60890]:     timer()
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  640.657585] env[60890]:     cb(*args, **kw)
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  640.657585] env[60890]:     result = function(*args, **kwargs)
[  640.657585] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  640.657585] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  640.657585] env[60890]:     return method(*args, **kwargs)
[  640.657585] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  640.657585] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  640.657585] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  640.657585] env[60890]:     result = self.transport._send(
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  640.657585] env[60890]:     return self._driver.send(target, ctxt, message,
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  640.657585] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  640.657585] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  640.657585] env[60890]:     raise result
[  640.657585] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  640.657585] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  640.716084] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.716505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.716505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.716663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.716802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.717084] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.717196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.717311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.717518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.717765] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.717912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.718023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.718223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.718359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.718500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.718816] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.718885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.718994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.719193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.719319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.719547] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.719694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.719805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.720012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.720144] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  640.720368] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9c24d5e63f6344f69254822be29db2f6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  640.725342] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9c24d5e63f6344f69254822be29db2f6
[  640.743909] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-bd857875-8626-4dca-adc6-652fd3699b66" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.744135] env[60890]: Traceback (most recent call last):
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  640.744191] env[60890]:     timer()
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  640.744191] env[60890]:     cb(*args, **kw)
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  640.744191] env[60890]:     result = function(*args, **kwargs)
[  640.744191] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  640.744191] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  640.744191] env[60890]:     return method(*args, **kwargs)
[  640.744191] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  640.744191] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  640.744191] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  640.744191] env[60890]:     result = self.transport._send(
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  640.744191] env[60890]:     return self._driver.send(target, ctxt, message,
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  640.744191] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  640.744191] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  640.744191] env[60890]:     raise result
[  640.744191] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  640.744191] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  640.995390] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.995598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.995729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.995998] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.996164] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.996428] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.996588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.996709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.996927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.997174] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.997328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.997448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.997657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.997804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.997954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.998204] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.998360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.998478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.998784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  640.998954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  640.999259] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  640.999535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  640.999641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  640.999860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.000027] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  641.000257] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 021e21e43ad74b19a0ba9c2bd258c83c in queue reply_156db6c4256a4c58884afce15e6f5340
[  641.007580] env[60890]: INFO networking_nsxv3.common.synchronization [None req-666c2e5d-b80a-4a78-9f1d-feb8bbffa159 None None] Enqueued Resource ID: 8176ef7a-d5b6-4974-a0dc-f77a08c8fffe with Priority: HIGHEST for action port
[  641.008098] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8176ef7a-d5b6-4974-a0dc-f77a08c8fffe with Priority: HIGHEST for action port
[  641.008598] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.008718] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.008928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.009140] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.009174] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.009316] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-8176ef7a-d5b6-4974-a0dc-f77a08c8fffe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.009422] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-8176ef7a-d5b6-4974-a0dc-f77a08c8fffe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.009489] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-8176ef7a-d5b6-4974-a0dc-f77a08c8fffe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.010424] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-8176ef7a-d5b6-4974-a0dc-f77a08c8fffe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.010424] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('8176ef7a-d5b6-4974-a0dc-f77a08c8fffe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  641.010424] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4314207289d247c6b6391e2774a471f4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  641.011148] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 021e21e43ad74b19a0ba9c2bd258c83c
[  641.014331] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4314207289d247c6b6391e2774a471f4
[  641.177853] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.178053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.178178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.178473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.178588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.178862] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.179013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.179130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.179340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.179575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.179724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.179837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.180050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.180191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.180334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.180576] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.180721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.180836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.181078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.181284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.181442] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.181590] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.181716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.181920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.182055] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  641.182284] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8462663cb7b14816bf17b55a7296e4c2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  641.188027] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8462663cb7b14816bf17b55a7296e4c2
[  641.222821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.223228] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.223228] env[60890]: ERROR oslo_messaging.rpc.server 
[  641.363089] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.363286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.363463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.363687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.363838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.364116] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.364277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.364398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.364617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.364928] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.365070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.365193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.365399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.365544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.365737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.365949] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.366102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.366221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.366425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.366609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.366786] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  641.366939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  641.367055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  641.367257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  641.367391] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  641.367622] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f412945fb5df42c888293b3eeb43dc41 in queue reply_156db6c4256a4c58884afce15e6f5340
[  641.375115] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f412945fb5df42c888293b3eeb43dc41
[  641.502433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.503093] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.503093] env[60890]: ERROR oslo_messaging.rpc.server 
[  641.511288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-8176ef7a-d5b6-4974-a0dc-f77a08c8fffe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.511449] env[60890]: Traceback (most recent call last):
[  641.511489] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  641.511489] env[60890]:     timer()
[  641.511489] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  641.511489] env[60890]:     cb(*args, **kw)
[  641.511489] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  641.511489] env[60890]:     result = function(*args, **kwargs)
[  641.511489] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  641.511489] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  641.511489] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  641.511489] env[60890]:     return method(*args, **kwargs)
[  641.511489] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  641.511489] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  641.511489] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  641.511489] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  641.511489] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  641.511489] env[60890]:     result = self.transport._send(
[  641.511993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  641.511993] env[60890]:     return self._driver.send(target, ctxt, message,
[  641.511993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  641.511993] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  641.511993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  641.511993] env[60890]:     raise result
[  641.511993] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.511993] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.686387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.686387] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.686387] env[60890]: ERROR oslo_messaging.rpc.server 
[  641.870639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.870793] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  641.870793] env[60890]: ERROR oslo_messaging.rpc.server 
[  642.097488] env[60890]: INFO networking_nsxv3.common.synchronization [req-106f5843-de9b-48dd-a8fc-31ea065d28c0 req-99dbe496-5d10-4d11-b310-13145557c504 tempest-ImagesOneServerNegativeTestJSON-629542534 tempest-ImagesOneServerNegativeTestJSON-629542534-project-member] Enqueued Resource ID: fc7c2101-440d-4f00-a078-693a6eccd3ac with Priority: HIGHEST for action security_group_members
[  642.098032] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: fc7c2101-440d-4f00-a078-693a6eccd3ac with Priority: HIGHEST for action security_group_members
[  642.098337] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.098458] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.098522] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.098924] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.098924] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.098924] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-fc7c2101-440d-4f00-a078-693a6eccd3ac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.099080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-fc7c2101-440d-4f00-a078-693a6eccd3ac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.099080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-fc7c2101-440d-4f00-a078-693a6eccd3ac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.099263] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-fc7c2101-440d-4f00-a078-693a6eccd3ac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.099386] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.099474] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.099529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.099686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.099762] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.099820] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('fc7c2101-440d-4f00-a078-693a6eccd3ac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  642.099983] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2a02980ba6154387874e136800a0a14c in queue reply_156db6c4256a4c58884afce15e6f5340
[  642.117596] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2a02980ba6154387874e136800a0a14c
[  642.131483] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.131483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.131483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.131483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.131483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.131483] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.131483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.131483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.131798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.131831] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.131965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.132097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.135128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.135128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.135128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.135128] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  642.135128] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4af2fdbcce67411f8586da0233dee52a in queue reply_156db6c4256a4c58884afce15e6f5340
[  642.142399] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4af2fdbcce67411f8586da0233dee52a
[  642.414108] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.414317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.414600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.414824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.414996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.415265] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.415428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.415563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.416059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.416059] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.416208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.416359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.416497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.416654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.416809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.417073] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.417224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.417345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.417555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.417693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.417935] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.418091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.418212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.418420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.418555] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  642.418792] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8a3c4307c17649829ade25e671b9f6c1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  642.425616] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8a3c4307c17649829ade25e671b9f6c1
[  642.449480] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.449689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.449921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.450091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.450243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.450504] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.450656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.450772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.451047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.451289] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.451440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.451555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.451760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.451896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.452052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.452302] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.452452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.452562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.452776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.452906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.453166] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.453349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.453428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.453684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.453859] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  642.454104] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 402a89f690984a279a0426254a8c10ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  642.455357] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.455526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.455685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.455923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.456118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.456373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.456531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.456652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.456904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.457101] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.457260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.457631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.457631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.457728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.457854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.458105] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.459760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.459760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.459760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.459760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.459760] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.459760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.459760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.459760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.459760] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  642.459760] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9e439dbd77564a94b25c4fdcbdd467af in queue reply_156db6c4256a4c58884afce15e6f5340
[  642.460981] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 402a89f690984a279a0426254a8c10ad
[  642.464996] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9e439dbd77564a94b25c4fdcbdd467af
[  642.603804] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.603804] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.603804] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: fc7c2101-440d-4f00-a078-693a6eccd3ac already deleted.
[  642.603804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-fc7c2101-440d-4f00-a078-693a6eccd3ac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.638423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  642.638423] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  642.638423] env[60890]: ERROR oslo_messaging.rpc.server 
[  642.920541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  642.921048] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  642.921048] env[60890]: ERROR oslo_messaging.rpc.server 
[  642.961361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  642.961909] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  642.961909] env[60890]: ERROR oslo_messaging.rpc.server 
[  642.963141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  642.963550] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  642.963550] env[60890]: ERROR oslo_messaging.rpc.server 
[  643.178735] env[60890]: INFO networking_nsxv3.common.synchronization [None req-7a9daa20-9216-4850-a905-019f1bc49146 None None] Enqueued Resource ID: f3b12e67-81e2-46ee-83bb-58085e7b5489 with Priority: HIGHEST for action port
[  643.179013] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f3b12e67-81e2-46ee-83bb-58085e7b5489 with Priority: HIGHEST for action port
[  643.179600] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.179732] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.179800] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.179994] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.180101] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.180248] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-f3b12e67-81e2-46ee-83bb-58085e7b5489 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.180339] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-f3b12e67-81e2-46ee-83bb-58085e7b5489" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.180399] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-f3b12e67-81e2-46ee-83bb-58085e7b5489" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.180600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-f3b12e67-81e2-46ee-83bb-58085e7b5489" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.180708] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f3b12e67-81e2-46ee-83bb-58085e7b5489',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  643.180847] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3884b8ffd41d43f7837a8d1121cf55b6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  643.201280] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3884b8ffd41d43f7837a8d1121cf55b6
[  643.255792] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.256030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.256170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.256438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.256586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.256858] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.257016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.257132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.257349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.257596] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.257749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.257869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.258077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.258223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.258370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.258618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.258772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.258890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.259096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.259229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.259465] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.259617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.259734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.259943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.260093] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  643.260326] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e470ee1855c74bb592984efca0940995 in queue reply_156db6c4256a4c58884afce15e6f5340
[  643.368593] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e470ee1855c74bb592984efca0940995
[  643.484112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.484112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.484112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.484112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.484112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.484112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.485342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.485342] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  643.485342] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 518a52e74aff4245897737b99932ed7b in queue reply_156db6c4256a4c58884afce15e6f5340
[  643.493810] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 518a52e74aff4245897737b99932ed7b
[  643.509980] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.510338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.510377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.510952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.511062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.511447] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.511502] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.511672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.511856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.512130] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.512295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.512419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.512629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.512878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.513060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.513316] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.513473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.513594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.513805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.513943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.514187] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.514345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.514464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.514670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.515125] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  643.515396] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 22461109ae1f4f289b864be04630e1fd in queue reply_156db6c4256a4c58884afce15e6f5340
[  643.523862] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 22461109ae1f4f289b864be04630e1fd
[  643.572444] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.572661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.572796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.573069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.573219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.573474] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.573630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.573749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.573962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.574207] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.574364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.574479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.574717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.574860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.575015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.575267] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.575453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.575626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.575901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.576897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.576897] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  643.576897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  643.576897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  643.576897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  643.576897] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  643.577168] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 851e022ee5c74d4597de04f8a0043ba1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  643.583617] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 851e022ee5c74d4597de04f8a0043ba1
[  643.682460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-f3b12e67-81e2-46ee-83bb-58085e7b5489" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.683178] env[60890]: Traceback (most recent call last):
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  643.683178] env[60890]:     timer()
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  643.683178] env[60890]:     cb(*args, **kw)
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  643.683178] env[60890]:     result = function(*args, **kwargs)
[  643.683178] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  643.683178] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  643.683178] env[60890]:     return method(*args, **kwargs)
[  643.683178] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  643.683178] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  643.683178] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  643.683178] env[60890]:     result = self.transport._send(
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  643.683178] env[60890]:     return self._driver.send(target, ctxt, message,
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  643.683178] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  643.683178] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  643.683178] env[60890]:     raise result
[  643.683178] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  643.683178] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  643.762170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  643.762721] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  643.762721] env[60890]: ERROR oslo_messaging.rpc.server 
[  643.991221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  643.991734] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  643.991734] env[60890]: ERROR oslo_messaging.rpc.server 
[  644.017741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  644.018274] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  644.018274] env[60890]: ERROR oslo_messaging.rpc.server 
[  644.079526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  644.080041] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  644.080041] env[60890]: ERROR oslo_messaging.rpc.server 
[  644.377989] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.378203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.378331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.378593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.378743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.379004] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.379185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.379261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.380027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.380027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.380027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.380027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.380255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.380385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.380534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.380786] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.380945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.381064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.381276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.381411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.381651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.381808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.381929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.382140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.382276] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  644.382512] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 58aeace2857748ec8121f8c4f7f21a7c in queue reply_156db6c4256a4c58884afce15e6f5340
[  644.388869] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 58aeace2857748ec8121f8c4f7f21a7c
[  644.560352] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.560554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.560683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.560963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.561813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.561813] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.561813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.561813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.562090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.562090] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.562247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.562430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.562657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.562809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.562963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.563217] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.563377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.563604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.563699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.563900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.564184] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.564339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.564455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.564660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.564828] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  644.565059] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f58ce170e604449cb2fadae08c9f7ed9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  644.571927] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.572117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.572243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.572467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.572604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.572862] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.573012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.573129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.573348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.573659] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.573832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.573956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.574172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.574315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.574466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.574747] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.574925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.575049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.575257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.575390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.575661] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.575803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.575973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.576158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.576296] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  644.576520] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 49886dbc72404d49aa7a4d55648e3a8e in queue reply_156db6c4256a4c58884afce15e6f5340
[  644.577805] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f58ce170e604449cb2fadae08c9f7ed9
[  644.583597] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 49886dbc72404d49aa7a4d55648e3a8e
[  644.688759] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.688969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.689094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.689358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.689503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.689801] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.689965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.690085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.690305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.690565] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.690692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.690812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.691025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.691233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.691491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.691761] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.691919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.692049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.692265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.692401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.692692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  644.692851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  644.692968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  644.693177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  644.693311] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  644.693543] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 85b8c451b12448c2878555a548ad02e5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  644.701972] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 85b8c451b12448c2878555a548ad02e5
[  644.884563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  644.885154] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  644.885154] env[60890]: ERROR oslo_messaging.rpc.server 
[  645.067236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  645.067812] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  645.067812] env[60890]: ERROR oslo_messaging.rpc.server 
[  645.078210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  645.078635] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  645.078635] env[60890]: ERROR oslo_messaging.rpc.server 
[  645.195612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  645.196206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  645.196206] env[60890]: ERROR oslo_messaging.rpc.server 
[  645.564945] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.565145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.565274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.565563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.565735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.565994] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-819 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.566152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.566270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.566488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.566729] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.566883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.567000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.567207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.567348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.567493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-819" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.567785] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.567908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.568018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.568236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.568369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.568603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.568756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.569116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.569345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.569484] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bf16a6de-7502-4b8f-b27b-33cb0ddfa26e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  645.569821] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 02bc61530ea64ec5a303cb3e0adda6c4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  645.576666] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 02bc61530ea64ec5a303cb3e0adda6c4
[  645.643956] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.644249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.644399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.644821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.644953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.645135] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.645283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.645404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.646065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.646065] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.646201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.646286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.646476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.646619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.646765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.647023] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.647174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.647290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.647493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.647626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.647861] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.648032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.648149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.648664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.648829] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  645.649066] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 388d9604430045c097e3edb407ea4e11 in queue reply_156db6c4256a4c58884afce15e6f5340
[  645.651370] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.651592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.651726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.651949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.652109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.652357] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.653489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.653489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.653489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.653489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.653489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.653489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.653702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.653702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.653816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.654072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.654226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.654346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.654555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.654734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.654954] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.655111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.655230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.655436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.655566] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  645.655928] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f4d3cdc14e8c41b88a8ee7af9d06df88 in queue reply_156db6c4256a4c58884afce15e6f5340
[  645.656645] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 388d9604430045c097e3edb407ea4e11
[  645.659615] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f4d3cdc14e8c41b88a8ee7af9d06df88
[  645.769995] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.770219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.770347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.770688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.770859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.771136] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.771277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.771394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.771613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.771874] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.772041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.772343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.772407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.772610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.772808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.773072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.773230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.773350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.773561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.773698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  645.773940] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  645.774093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  645.774211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  645.774419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  645.774559] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  645.774821] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3b0c91c35a9c499fbd84407142a0d05e in queue reply_156db6c4256a4c58884afce15e6f5340
[  645.780566] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3b0c91c35a9c499fbd84407142a0d05e
[  646.073434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-bf16a6de-7502-4b8f-b27b-33cb0ddfa26e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  646.073958] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  646.073958] env[60890]: ERROR oslo_messaging.rpc.server 
[  646.151308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  646.151835] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  646.151835] env[60890]: ERROR oslo_messaging.rpc.server 
[  646.157719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  646.158152] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  646.158152] env[60890]: ERROR oslo_messaging.rpc.server 
[  646.277736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  646.278239] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  646.278239] env[60890]: ERROR oslo_messaging.rpc.server 
[  646.438110] env[60890]: INFO networking_nsxv3.common.synchronization [req-1b1d843e-1b51-4030-b1d1-4b2e0d4e78c0 req-c557abd4-0342-4761-80a1-51d573a21c73 tempest-ListServersNegativeTestJSON-955562437 tempest-ListServersNegativeTestJSON-955562437-project-member] Enqueued Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  646.438110] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 with Priority: HIGHEST for action security_group_members
[  646.438110] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.438110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.438110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.438110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.438110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.438110] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-976f3f29-8ecd-43ae-ac0b-7b7127db7112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.438110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.438110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.438612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.438612] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.438612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.438612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.438738] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.439304] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.439304] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('976f3f29-8ecd-43ae-ac0b-7b7127db7112',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  646.439304] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f6ccb604b15a41cb9a09f53ffd0008ac in queue reply_156db6c4256a4c58884afce15e6f5340
[  646.449490] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f6ccb604b15a41cb9a09f53ffd0008ac
[  646.773955] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.774147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.774273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.774551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.774715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.775016] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.775176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.775296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.775532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.775775] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.775926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.776054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.776313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.776458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.777171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.777171] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.777171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.777171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.777640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.777640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.777859] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.777911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.778221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.778436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.778569] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  646.778796] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d0dfdde8fa064accb94de0b594a7a619 in queue reply_156db6c4256a4c58884afce15e6f5340
[  646.785914] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d0dfdde8fa064accb94de0b594a7a619
[  646.793283] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.793460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.793582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.793826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.793962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.794222] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.794426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.794551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.794796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.795045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.795201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.795318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.795523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.795671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.802990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.803284] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.803449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.803569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.803798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.803936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.804193] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.804347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.804463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.804669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.805100] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  646.805352] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8e5923dec809407bb365a1a0115f0dde in queue reply_156db6c4256a4c58884afce15e6f5340
[  646.812549] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8e5923dec809407bb365a1a0115f0dde
[  646.850456] env[60890]: INFO networking_nsxv3.common.synchronization [req-6546bc9e-998c-443f-ae52-ec99c0502841 req-109c4bec-aeac-42bf-86d0-ce73baf0e307 tempest-ServersTestBootFromVolume-1482275162 tempest-ServersTestBootFromVolume-1482275162-project-member] Enqueued Resource ID: e0852405-c71e-4825-ac71-a12a84643215 with Priority: HIGHEST for action security_group_members
[  646.851128] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e0852405-c71e-4825-ac71-a12a84643215 with Priority: HIGHEST for action security_group_members
[  646.851464] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.851579] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.851639] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.851865] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.851949] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.856646] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e0852405-c71e-4825-ac71-a12a84643215 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.856646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.856646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.856646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.856646] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.856646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.856646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.856646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.856646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.856646] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e0852405-c71e-4825-ac71-a12a84643215',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  646.856646] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b4624e188a1c4e46a46ae0aaf01bc5e7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  646.861506] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b4624e188a1c4e46a46ae0aaf01bc5e7
[  646.943458] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.943600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.943701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.943880] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.943974] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.944185] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.944282] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.944385] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.944600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.944600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.944701] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 976f3f29-8ecd-43ae-ac0b-7b7127db7112 already deleted.
[  646.944742] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-976f3f29-8ecd-43ae-ac0b-7b7127db7112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.994624] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.994866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.995004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.995262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.995411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.995677] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.995833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.995954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.996183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.996428] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.996582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.996700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.996912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.997072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.997239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.997489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.997643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.997763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.997968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.998103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  646.998336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  646.998489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  646.998607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  646.998857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  646.998941] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  646.999215] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fcc5253f9b514fa6aaf269c2525913d4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  647.010711] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fcc5253f9b514fa6aaf269c2525913d4
[  647.135087] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  647.135371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  647.135417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  647.135673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  647.135843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.136124] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  647.136286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  647.136406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  647.136702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  647.136938] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  647.137094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  647.137212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  647.137416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  647.137557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.137702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.137950] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  647.138248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  647.138248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  647.138482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  647.138544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.138825] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  647.138977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  647.139093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  647.140182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  647.140182] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  647.140182] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c0248d1bf85b467b809887b73c09333c in queue reply_156db6c4256a4c58884afce15e6f5340
[  647.145843] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c0248d1bf85b467b809887b73c09333c
[  647.281231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  647.282510] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  647.282510] env[60890]: ERROR oslo_messaging.rpc.server 
[  647.307897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  647.308440] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  647.308440] env[60890]: ERROR oslo_messaging.rpc.server 
[  647.355140] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  647.355294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  647.355367] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  647.355610] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  647.355736] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.355962] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  647.356082] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  647.356145] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  647.356300] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  647.356379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.356465] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e0852405-c71e-4825-ac71-a12a84643215 already deleted.
[  647.356543] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.500866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  647.501396] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  647.501396] env[60890]: ERROR oslo_messaging.rpc.server 
[  647.642205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  647.642805] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  647.642805] env[60890]: ERROR oslo_messaging.rpc.server 
[  648.099753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.099959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.100111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.100386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.100537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.100795] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.100957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.101091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.101312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.101558] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.101716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.101838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.102048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.102194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.102348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.102612] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.102797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.102927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.103140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.103278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.103520] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.104551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.104551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.104722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.104878] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d3b111dd-dbc2-4e22-86b9-9bee99c8cb12',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  648.105119] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ddf628d6cf624a28abddfaf623fb2c4b in queue reply_156db6c4256a4c58884afce15e6f5340
[  648.107307] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.107482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.107785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.107843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.107948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.108213] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.108369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.108491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.110330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.110330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.110330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.110931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.110931] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.110931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.111098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.111152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.111419] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  648.111492] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cd211b27136245089966d101dea8a88d in queue reply_156db6c4256a4c58884afce15e6f5340
[  648.112741] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ddf628d6cf624a28abddfaf623fb2c4b
[  648.116373] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cd211b27136245089966d101dea8a88d
[  648.143652] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.144258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.144258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.144492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.144696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.144983] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.145148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.145271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.145733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.145998] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.146160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.146283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.146535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.146638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.146789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.147060] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.147243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.147366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.147580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.147719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.147963] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.148331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.148331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.148455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.148586] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  648.148820] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg feaff15f86b74ac9bfbfdca9323c61f9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  648.154355] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg feaff15f86b74ac9bfbfdca9323c61f9
[  648.247129] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.247363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.247488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.247760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.247924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.248210] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.248366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.248481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.248702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.248977] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.249152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.249266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.249480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.249607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.249764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.250018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.250168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.250283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.250486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.250615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.250872] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.251037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.251157] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.251367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.251503] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  648.251725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7bbf3505e1634118bc7b8f325f1e7554 in queue reply_156db6c4256a4c58884afce15e6f5340
[  648.256908] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7bbf3505e1634118bc7b8f325f1e7554
[  648.607225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d3b111dd-dbc2-4e22-86b9-9bee99c8cb12" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  648.607746] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  648.607746] env[60890]: ERROR oslo_messaging.rpc.server 
[  648.614878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  648.615338] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  648.615338] env[60890]: ERROR oslo_messaging.rpc.server 
[  648.651358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  648.651863] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  648.651863] env[60890]: ERROR oslo_messaging.rpc.server 
[  648.739756] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.740266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.740539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.740876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.741093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.741421] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.741680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.741940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.742188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.742497] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.742751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.742957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.743247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.743453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.743781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.744290] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.744608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.744844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.745121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.745321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.745639] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  648.745874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  648.746048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  648.746315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  648.746518] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  648.746809] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6f3a5ac251d04b90841f90da62cdd41c in queue reply_156db6c4256a4c58884afce15e6f5340
[  648.751826] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6f3a5ac251d04b90841f90da62cdd41c
[  648.753433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  648.753992] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  648.753992] env[60890]: ERROR oslo_messaging.rpc.server 
[  649.220334] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.220334] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.220334] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.220334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.222827] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.222827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.222827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.223261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.223841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.224119] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.224486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.224719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.225111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.232259] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  649.232621] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a1b0a62fc8914947825dcc503cd0260b in queue reply_156db6c4256a4c58884afce15e6f5340
[  649.239856] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a1b0a62fc8914947825dcc503cd0260b
[  649.250004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  649.250601] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  649.250601] env[60890]: ERROR oslo_messaging.rpc.server 
[  649.301644] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.302064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.302329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.302720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.303011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.303529] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.303820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.304083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.304425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.304897] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.305211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.305462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.305795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.306093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.306472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.306845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.315587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.315587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.315587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.315717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.315972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.316168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.316291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.316519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.316657] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  649.316896] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 914295bf77ef4c96acbef4a3d22d4fac in queue reply_156db6c4256a4c58884afce15e6f5340
[  649.325465] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 914295bf77ef4c96acbef4a3d22d4fac
[  649.735263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  649.735805] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  649.735805] env[60890]: ERROR oslo_messaging.rpc.server 
[  649.821143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  649.821660] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  649.821660] env[60890]: ERROR oslo_messaging.rpc.server 
[  649.847227] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.847435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.847566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.847847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.848050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.848288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.848451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.848574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.848798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.849048] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.849204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.849325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.849536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.849682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.849838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.850094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.850252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.850369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.850579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.850719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  649.850960] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  649.851116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  649.851235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  649.851443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  649.851617] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  649.851804] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg db4d970abb504d76acb26a71e9bd7dc4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  649.862530] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg db4d970abb504d76acb26a71e9bd7dc4
[  650.068199] env[60890]: INFO networking_nsxv3.common.synchronization [req-b6d8a71d-b17f-45e2-abce-d7f249060f28 req-2f4f9d0d-3479-4e69-a233-776a4c2047b0 tempest-MigrationsAdminTest-320299187 tempest-MigrationsAdminTest-320299187-project-member] Enqueued Resource ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b with Priority: HIGHEST for action security_group_members
[  650.068712] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b with Priority: HIGHEST for action security_group_members
[  650.069215] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.069215] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.069215] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.069412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.069496] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.069648] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-18ba6c5b-4074-4f30-b957-61d36273ed7b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.069775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.069807] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.069943] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.070092] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.070234] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.070335] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.070551] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.070676] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.070776] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('18ba6c5b-4074-4f30-b957-61d36273ed7b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  650.072214] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg eb1cd303a73d40e7bb61cc99a55df7cf in queue reply_156db6c4256a4c58884afce15e6f5340
[  650.083406] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eb1cd303a73d40e7bb61cc99a55df7cf
[  650.280170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.280170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-630 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.280170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-630" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.280170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.280170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.280170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.280170] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65d957b9-8b90-4e30-9870-0b3bd44f1dc7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  650.281907] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e0bab1c5982949d4b56f2f9e140ad325 in queue reply_156db6c4256a4c58884afce15e6f5340
[  650.287553] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e0bab1c5982949d4b56f2f9e140ad325
[  650.353193] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.353396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.353526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.353799] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.353953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.354299] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.354475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.354597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.354854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.355104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.355259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.355378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.355589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.355738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.355889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.356170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.356331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.356450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.356658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.356795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.357156] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.357273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.357412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.357627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.357767] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  650.358006] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c8c61f464ce1481dbafd1b6c0c3652e4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  650.358876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  650.359329] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  650.359329] env[60890]: ERROR oslo_messaging.rpc.server 
[  650.366301] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c8c61f464ce1481dbafd1b6c0c3652e4
[  650.574510] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.574671] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.574758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.575106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.575106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.575312] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.575385] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.575422] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.575571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.575650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.575736] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 18ba6c5b-4074-4f30-b957-61d36273ed7b already deleted.
[  650.575835] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-18ba6c5b-4074-4f30-b957-61d36273ed7b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.786391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-65d957b9-8b90-4e30-9870-0b3bd44f1dc7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  650.786915] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  650.786915] env[60890]: ERROR oslo_messaging.rpc.server 
[  650.870153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  650.871050] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  650.871050] env[60890]: ERROR oslo_messaging.rpc.server 
[  650.904132] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.904349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.904478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.904803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.904962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.905224] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.905376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.905493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.905982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.906231] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.906408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.906533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.906765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.906910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.907058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.907313] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.907463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.907611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.907835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.907966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  650.908225] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  650.908373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  650.908488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  650.908704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  650.908839] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  650.909071] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0b6cef3c6b7b44c08d2c7ab41b7cfd5c in queue reply_156db6c4256a4c58884afce15e6f5340
[  650.916027] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0b6cef3c6b7b44c08d2c7ab41b7cfd5c
[  651.394956] env[60890]: INFO networking_nsxv3.common.synchronization [req-05ecc47b-1e49-4859-90a9-ffeabd15eafe req-c9cf6d7c-f030-49b8-a5f2-c7135dc0830e tempest-ServerTagsTestJSON-644809188 tempest-ServerTagsTestJSON-644809188-project-member] Enqueued Resource ID: 8e84b2b5-833c-4fe0-b63b-f1d9445f086f with Priority: HIGHEST for action security_group_members
[  651.395477] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8e84b2b5-833c-4fe0-b63b-f1d9445f086f with Priority: HIGHEST for action security_group_members
[  651.395811] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.395925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.395993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.396217] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.396298] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.396449] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.396543] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.396603] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.396789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.397008] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.397008] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.397108] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.397362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.397362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.397362] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('8e84b2b5-833c-4fe0-b63b-f1d9445f086f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  651.397523] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7cb8906143d74252a652c4909a9add95 in queue reply_156db6c4256a4c58884afce15e6f5340
[  651.412269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  651.412625] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  651.412625] env[60890]: ERROR oslo_messaging.rpc.server 
[  651.417227] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7cb8906143d74252a652c4909a9add95
[  651.465360] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.465553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.465680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.465972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.466133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.466657] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.466657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.466808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.466904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.467137] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.467295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.467413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.467649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.467813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.467966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.468230] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.468378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.468492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.468701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.468837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.469120] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.469279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.469399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.469605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.469735] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  651.469961] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4e14cfb54520477ebce877867eaf2fab in queue reply_156db6c4256a4c58884afce15e6f5340
[  651.476891] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4e14cfb54520477ebce877867eaf2fab
[  651.631409] env[60890]: INFO networking_nsxv3.common.synchronization [req-1ab9a664-6cb8-44e9-9561-473bf8192aaf req-66949d8a-e120-4568-a55b-0d24e9ece481 tempest-ServerExternalEventsTest-384691146 tempest-ServerExternalEventsTest-384691146-project-member] Enqueued Resource ID: 7501ba73-c2dc-466a-a7ca-3bc2146756d9 with Priority: HIGHEST for action security_group_members
[  651.631904] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7501ba73-c2dc-466a-a7ca-3bc2146756d9 with Priority: HIGHEST for action security_group_members
[  651.632276] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.632433] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.632506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.632729] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.640062] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.640062] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7501ba73-c2dc-466a-a7ca-3bc2146756d9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.640062] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7501ba73-c2dc-466a-a7ca-3bc2146756d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.640062] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7501ba73-c2dc-466a-a7ca-3bc2146756d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.640062] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7501ba73-c2dc-466a-a7ca-3bc2146756d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.640353] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.640353] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.640353] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.640477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.640477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.640546] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7501ba73-c2dc-466a-a7ca-3bc2146756d9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  651.641266] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg dbf2534ae8574869b3987250bae8fed4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  651.648503] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dbf2534ae8574869b3987250bae8fed4
[  651.768565] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.768772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.768903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.769170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.769319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.769599] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.769723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.769844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.776746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.777028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.777213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.777337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.777556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.777704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.777857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.778114] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.778254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.778372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.778579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.778714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.780023] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.780023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.780023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.780023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.780023] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  651.780023] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fb57911aee4e487bba1287785426cfc6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  651.784603] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fb57911aee4e487bba1287785426cfc6
[  651.905136] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.905296] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.905371] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.905583] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.905678] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.905868] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.905985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.906026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.906299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.906299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.906364] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 8e84b2b5-833c-4fe0-b63b-f1d9445f086f already deleted.
[  651.906406] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.957037] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.957230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.957352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.957634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.957785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.958043] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-571 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.958195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.958310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.958524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.958894] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.958995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.959077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.959279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.959419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.959563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-571" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.959807] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.959954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.960162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.960385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.960516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.960762] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-a94f24cb-11b4-4a23-9915-6ed7998515f2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  651.960911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  651.961027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  651.961232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  651.961364] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('a94f24cb-11b4-4a23-9915-6ed7998515f2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  651.961588] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 734e50cc1641404183fa023994953abf in queue reply_156db6c4256a4c58884afce15e6f5340
[  651.968545] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 734e50cc1641404183fa023994953abf
[  651.972860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  651.973350] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  651.973350] env[60890]: ERROR oslo_messaging.rpc.server 
[  652.142498] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.142752] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.142871] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.143151] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.143283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.143550] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.143685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.143790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.143970] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.144094] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.144331] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7501ba73-c2dc-466a-a7ca-3bc2146756d9 already deleted.
[  652.144505] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7501ba73-c2dc-466a-a7ca-3bc2146756d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.283835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  652.283835] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  652.283835] env[60890]: ERROR oslo_messaging.rpc.server 
[  652.463491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-a94f24cb-11b4-4a23-9915-6ed7998515f2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  652.464066] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  652.464066] env[60890]: ERROR oslo_messaging.rpc.server 
[  652.514843] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.515059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.515191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.515465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.515614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.515901] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.516077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.516204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.516424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.517228] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.517403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.517530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.520033] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.520033] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.520033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.520033] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  652.520033] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e4f087b408754aab8673fdde48af0e2f in queue reply_156db6c4256a4c58884afce15e6f5340
[  652.525191] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e4f087b408754aab8673fdde48af0e2f
[  652.873267] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.873563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.873627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.873908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.874059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.874340] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.874499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.874620] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.874882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.875133] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.875291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.875412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.875618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.875762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.875911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.876189] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.876348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.876463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.876668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.876802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  652.877042] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  652.877193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  652.877310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  652.877670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  652.877670] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  652.877868] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b47489914442415eb2994eeb11984add in queue reply_156db6c4256a4c58884afce15e6f5340
[  652.886557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b47489914442415eb2994eeb11984add
[  653.021947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  653.022441] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  653.022441] env[60890]: ERROR oslo_messaging.rpc.server 
[  653.380118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  653.380683] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  653.380683] env[60890]: ERROR oslo_messaging.rpc.server 
[  653.472185] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  653.472483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  653.472523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  653.472794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  653.472946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.473222] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  653.473395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  653.473868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  653.473868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  653.474023] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  653.474130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  653.474228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  653.474422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  653.474572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.474747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.475014] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  653.475173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  653.475296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  653.475859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  653.476068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.476744] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  653.476954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  653.477088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  653.477312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  653.477456] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  653.477813] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a4bd774d091c4d85baf730b9f1dd3315 in queue reply_156db6c4256a4c58884afce15e6f5340
[  653.484080] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a4bd774d091c4d85baf730b9f1dd3315
[  653.655987] env[60890]: INFO networking_nsxv3.common.synchronization [req-106f5843-de9b-48dd-a8fc-31ea065d28c0 req-b689f59c-44ce-4acc-8ed8-44d1dc3cbe6f tempest-ImagesOneServerNegativeTestJSON-629542534 tempest-ImagesOneServerNegativeTestJSON-629542534-project-member] Enqueued Resource ID: fc7c2101-440d-4f00-a078-693a6eccd3ac with Priority: HIGHEST for action security_group_members
[  653.657414] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: fc7c2101-440d-4f00-a078-693a6eccd3ac with Priority: HIGHEST for action security_group_members
[  653.657414] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  653.657414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  653.657414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  653.657414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  653.657414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.657695] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-fc7c2101-440d-4f00-a078-693a6eccd3ac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  653.657695] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-fc7c2101-440d-4f00-a078-693a6eccd3ac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  653.657695] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-fc7c2101-440d-4f00-a078-693a6eccd3ac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  653.657797] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-fc7c2101-440d-4f00-a078-693a6eccd3ac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  653.657925] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  653.658018] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  653.658103] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  653.658215] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  653.658292] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.658381] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('fc7c2101-440d-4f00-a078-693a6eccd3ac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  653.658557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d0f55fb83d224b0f8d4affdd02a16ddc in queue reply_156db6c4256a4c58884afce15e6f5340
[  653.669300] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d0f55fb83d224b0f8d4affdd02a16ddc
[  653.980049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  653.980712] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  653.980712] env[60890]: ERROR oslo_messaging.rpc.server 
[  654.002008] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.002212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.002212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.002469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.002642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.002921] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.003084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.003204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.003431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.003675] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.003831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.003949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.004187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.004336] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.004484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.004765] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.004971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.005097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.005309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.005446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.011482] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.011482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.011482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.011482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.011482] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  654.011482] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ef5d233fec3d4cbb9c15105b58e91387 in queue reply_156db6c4256a4c58884afce15e6f5340
[  654.015459] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ef5d233fec3d4cbb9c15105b58e91387
[  654.160600] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.160754] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.160822] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.161040] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.161122] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.161319] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.161412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.161483] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.161629] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.161701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.161782] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: fc7c2101-440d-4f00-a078-693a6eccd3ac already deleted.
[  654.161860] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-fc7c2101-440d-4f00-a078-693a6eccd3ac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.453242] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0c5b2b3d2bf348ca837fac5a7b13b466 in queue reply_156db6c4256a4c58884afce15e6f5340
[  654.465145] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0c5b2b3d2bf348ca837fac5a7b13b466
[  654.511583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  654.512118] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  654.512118] env[60890]: ERROR oslo_messaging.rpc.server 
[  654.521982] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.522191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.522320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.522584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.522734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.523018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.523337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.523472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.523697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.523958] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.524128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.524253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.524485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.524631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.524804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.525063] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.525224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.525346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.525598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.525719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  654.526006] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  654.526168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  654.526288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  654.526494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  654.526629] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  654.526852] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 328f094c613d4079b7f614491b29fa55 in queue reply_156db6c4256a4c58884afce15e6f5340
[  654.531845] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 328f094c613d4079b7f614491b29fa55
[  655.028870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  655.029401] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  655.029401] env[60890]: ERROR oslo_messaging.rpc.server 
[  655.145905] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.145905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.145905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.145905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.146232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.146232] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.147172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.147172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.147172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.147172] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.147172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.147537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.152288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.152454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.152619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.152890] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.153055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.153182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.154013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.154171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.154426] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.154697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.154862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.155091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.155408] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  655.156568] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a706293258e949c2a28f069e3df01d32 in queue reply_156db6c4256a4c58884afce15e6f5340
[  655.163474] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a706293258e949c2a28f069e3df01d32
[  655.599360] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.599702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.599702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.600092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.600251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.600552] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.600717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.600856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.601187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.601346] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.601494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.601617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.601830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.601977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.602132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.602383] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.602537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.602657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.602884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.603046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.603288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  655.603442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  655.603561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  655.603768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  655.603928] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  655.604318] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7e6c31120e9b4dd69ecefdceae2f34e7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  655.610922] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7e6c31120e9b4dd69ecefdceae2f34e7
[  655.657241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  655.657743] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  655.657743] env[60890]: ERROR oslo_messaging.rpc.server 
[  656.106129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  656.106658] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  656.106658] env[60890]: ERROR oslo_messaging.rpc.server 
[  656.265870] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.266077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.266205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.266468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.266613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.266873] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.267033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.267152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.267373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.267615] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.267766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.267884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.268122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.268272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.268537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.268794] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.268951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.269069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.269280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.269414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.269650] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.269807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.269928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.270136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.270274] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  656.270502] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 73be121991cd409fb8edf0a1615f2316 in queue reply_156db6c4256a4c58884afce15e6f5340
[  656.277637] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 73be121991cd409fb8edf0a1615f2316
[  656.661590] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.661884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.661939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.662189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.662336] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.662598] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-112 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.662769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.663047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.663109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.663352] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.663507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.663624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.663832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.663979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.664151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-112" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.664400] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.664553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.664671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.664989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.665144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.665386] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-264d6010-9da2-42db-aaf8-6eab3066f018 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  656.665543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  656.665696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  656.665895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  656.666077] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('264d6010-9da2-42db-aaf8-6eab3066f018',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  656.666266] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 714120469d5248288d2da4ae02bb8953 in queue reply_156db6c4256a4c58884afce15e6f5340
[  656.672252] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 714120469d5248288d2da4ae02bb8953
[  656.773133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  656.773660] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  656.773660] env[60890]: ERROR oslo_messaging.rpc.server 
[  657.168951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-264d6010-9da2-42db-aaf8-6eab3066f018" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  657.169465] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  657.169465] env[60890]: ERROR oslo_messaging.rpc.server 
[  657.228036] env[60890]: INFO networking_nsxv3.common.synchronization [None req-57d4c7e6-7db2-4659-a6be-a8e689505a0f None None] Enqueued Resource ID: 6fc73e1f-8285-432c-a230-e1b26505466c with Priority: HIGHEST for action port
[  657.228314] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6fc73e1f-8285-432c-a230-e1b26505466c with Priority: HIGHEST for action port
[  657.228882] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.229015] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.229085] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.229278] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.229364] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.229513] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-6fc73e1f-8285-432c-a230-e1b26505466c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.229607] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-6fc73e1f-8285-432c-a230-e1b26505466c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.229669] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-6fc73e1f-8285-432c-a230-e1b26505466c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.229865] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-6fc73e1f-8285-432c-a230-e1b26505466c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.229949] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6fc73e1f-8285-432c-a230-e1b26505466c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  657.230123] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a3e54de76f0b4c55a2a09ddc61b3de81 in queue reply_156db6c4256a4c58884afce15e6f5340
[  657.245064] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a3e54de76f0b4c55a2a09ddc61b3de81
[  657.602051] env[60890]: INFO networking_nsxv3.common.synchronization [req-077777b7-939e-443c-9ab8-671039fc4360 req-0738621b-53c1-4d5e-a53f-59e941aa723f tempest-VolumesAdminNegativeTest-2086354477 tempest-VolumesAdminNegativeTest-2086354477-project-member] Enqueued Resource ID: a4acf57a-8478-4dd9-b673-474172bcc683 with Priority: HIGHEST for action security_group_members
[  657.602576] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a4acf57a-8478-4dd9-b673-474172bcc683 with Priority: HIGHEST for action security_group_members
[  657.602960] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.603111] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.603186] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.603400] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.603520] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.603681] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-a4acf57a-8478-4dd9-b673-474172bcc683 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.603783] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-a4acf57a-8478-4dd9-b673-474172bcc683" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.603844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-a4acf57a-8478-4dd9-b673-474172bcc683" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.604063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-a4acf57a-8478-4dd9-b673-474172bcc683" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.608106] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.608106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.608106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.608106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.608106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.608106] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('a4acf57a-8478-4dd9-b673-474172bcc683',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  657.608106] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg fdc4c19a9a9c45a0b1d55b9a4dad2406 in queue reply_156db6c4256a4c58884afce15e6f5340
[  657.616257] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fdc4c19a9a9c45a0b1d55b9a4dad2406
[  657.732970] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-6fc73e1f-8285-432c-a230-e1b26505466c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.733260] env[60890]: Traceback (most recent call last):
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  657.733260] env[60890]:     timer()
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  657.733260] env[60890]:     cb(*args, **kw)
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  657.733260] env[60890]:     result = function(*args, **kwargs)
[  657.733260] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  657.733260] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  657.733260] env[60890]:     return method(*args, **kwargs)
[  657.733260] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  657.733260] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  657.733260] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  657.733260] env[60890]:     result = self.transport._send(
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  657.733260] env[60890]:     return self._driver.send(target, ctxt, message,
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  657.733260] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  657.733260] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  657.733260] env[60890]:     raise result
[  657.733921] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  657.733921] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  657.853971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.854178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.854308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.854580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.854758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.855030] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.855191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.855310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.855537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.855828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.856087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.856150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.856354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.856503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.856655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.856907] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.857066] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.857187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.857397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.857613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  657.860259] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  657.860259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  657.860259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  657.860259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  657.860259] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  657.860259] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 91f1f5de0efb46989b4491dfda1d9527 in queue reply_156db6c4256a4c58884afce15e6f5340
[  657.866829] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 91f1f5de0efb46989b4491dfda1d9527
[  658.115130] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.115282] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.115355] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.115580] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.115691] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.115893] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.116000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.116076] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.116233] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.116314] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.116399] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: a4acf57a-8478-4dd9-b673-474172bcc683 already deleted.
[  658.116477] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-a4acf57a-8478-4dd9-b673-474172bcc683" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.149926] env[60890]: INFO networking_nsxv3.common.synchronization [req-6546bc9e-998c-443f-ae52-ec99c0502841 req-6fde26de-7da3-46cf-8453-4b066912feb9 tempest-ServersTestBootFromVolume-1482275162 tempest-ServersTestBootFromVolume-1482275162-project-member] Enqueued Resource ID: e0852405-c71e-4825-ac71-a12a84643215 with Priority: HIGHEST for action security_group_members
[  658.149926] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e0852405-c71e-4825-ac71-a12a84643215 with Priority: HIGHEST for action security_group_members
[  658.149926] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.149926] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e0852405-c71e-4825-ac71-a12a84643215 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.149926] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.149926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.149926] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e0852405-c71e-4825-ac71-a12a84643215',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  658.149926] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg eef340b30ac34dcf988cbf4f63524038 in queue reply_156db6c4256a4c58884afce15e6f5340
[  658.163796] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eef340b30ac34dcf988cbf4f63524038
[  658.260051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.260051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.260051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.260051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.260051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.260051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.260051] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  658.260051] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ae7f226dc16848d287ddac46919ee0fe in queue reply_156db6c4256a4c58884afce15e6f5340
[  658.272662] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ae7f226dc16848d287ddac46919ee0fe
[  658.362674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  658.363265] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  658.363265] env[60890]: ERROR oslo_messaging.rpc.server 
[  658.589613] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.589817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.589947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.590221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.590368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.590618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.590775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.590897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.591111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.591409] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.591617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.591744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.591957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.592118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.592270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.592521] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.592678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.592798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.593008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.593143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.593382] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.593535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.593655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.593862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.593997] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  658.594227] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 878786f89ede408a81747b54375ffeda in queue reply_156db6c4256a4c58884afce15e6f5340
[  658.599554] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 878786f89ede408a81747b54375ffeda
[  658.649642] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.649806] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.649894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.650097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.650195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.650394] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  658.650493] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  658.650552] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  658.650701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  658.650779] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.650862] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e0852405-c71e-4825-ac71-a12a84643215 already deleted.
[  658.650941] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e0852405-c71e-4825-ac71-a12a84643215" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.762466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  658.763002] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  658.763002] env[60890]: ERROR oslo_messaging.rpc.server 
[  659.096450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  659.096959] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  659.096959] env[60890]: ERROR oslo_messaging.rpc.server 
[  659.354288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.354529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.354660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.354949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.355095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.355355] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.355505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.355620] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.355997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.356171] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.356328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.356444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.356646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.356785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.356931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.357265] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.357453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.357573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.357781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.357914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.358148] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.358295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.358410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.358610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.358764] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  659.359013] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 807831c35f394dc6aa5d499635efb6aa in queue reply_156db6c4256a4c58884afce15e6f5340
[  659.364765] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 807831c35f394dc6aa5d499635efb6aa
[  659.446897] env[60890]: INFO networking_nsxv3.common.synchronization [None req-75397bd5-f018-4a26-858a-2c12becb4eb4 None None] Enqueued Resource ID: 55ca08a0-932c-4e79-852e-c8a4a91b1802 with Priority: HIGHEST for action port
[  659.447148] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 55ca08a0-932c-4e79-852e-c8a4a91b1802 with Priority: HIGHEST for action port
[  659.447717] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.447830] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.447928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.448120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.448220] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.448343] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-55ca08a0-932c-4e79-852e-c8a4a91b1802 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.448440] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-55ca08a0-932c-4e79-852e-c8a4a91b1802" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.448498] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-55ca08a0-932c-4e79-852e-c8a4a91b1802" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.448691] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-55ca08a0-932c-4e79-852e-c8a4a91b1802" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.448807] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('55ca08a0-932c-4e79-852e-c8a4a91b1802',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  659.448937] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 97f8f7c3e9594307b05c3497dee9a46d in queue reply_156db6c4256a4c58884afce15e6f5340
[  659.458880] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 97f8f7c3e9594307b05c3497dee9a46d
[  659.687595] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.687821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.687952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.688274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.688426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.688689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.688848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.688970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.689193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.689440] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.689598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.689720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.689929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.690074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.690223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.690473] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.690630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.690750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.690960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.691096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.691335] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.691488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.691607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.691815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.691949] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  659.692226] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 52de596c30a14c3b8168e211a36ea42b in queue reply_156db6c4256a4c58884afce15e6f5340
[  659.697614] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 52de596c30a14c3b8168e211a36ea42b
[  659.861139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  659.861649] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  659.861649] env[60890]: ERROR oslo_messaging.rpc.server 
[  659.931775] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.931994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.932157] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.932433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.932587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.932849] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.933003] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.933121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.933332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.933577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.933731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.933847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.934052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.934193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.934339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.934593] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.934795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.934926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.935140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.935272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.935511] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  659.935663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  659.935781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  659.936038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  659.936180] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  659.936419] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 472559b5bec849738b96845bceadcbf5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  659.942033] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 472559b5bec849738b96845bceadcbf5
[  659.950179] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-55ca08a0-932c-4e79-852e-c8a4a91b1802" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  659.950347] env[60890]: Traceback (most recent call last):
[  659.950381] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  659.950381] env[60890]:     timer()
[  659.950381] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  659.950381] env[60890]:     cb(*args, **kw)
[  659.950381] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  659.950381] env[60890]:     result = function(*args, **kwargs)
[  659.950381] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  659.950381] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  659.950604] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  659.950604] env[60890]:     return method(*args, **kwargs)
[  659.950604] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  659.950604] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  659.950604] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  659.950604] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  659.950604] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  659.950604] env[60890]:     result = self.transport._send(
[  659.950604] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  659.950604] env[60890]:     return self._driver.send(target, ctxt, message,
[  659.950604] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  659.950604] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  659.950604] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  659.950604] env[60890]:     raise result
[  659.950604] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  659.950604] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  660.194207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  660.194769] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  660.194769] env[60890]: ERROR oslo_messaging.rpc.server 
[  660.438997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  660.439518] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  660.439518] env[60890]: ERROR oslo_messaging.rpc.server 
[  660.763149] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.763149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.763149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.763364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.763364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.764118] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.764118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.764118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.764118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.764502] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.764595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.764636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.765578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.765578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.765578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.765578] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.765578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.765849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.765888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.766113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.766269] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.766420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.766536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.766742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.766873] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  660.767109] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7abfee89411a4a88bbf581fa4068fd60 in queue reply_156db6c4256a4c58884afce15e6f5340
[  660.772931] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7abfee89411a4a88bbf581fa4068fd60
[  660.978503] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.978818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.978881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.979161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.979320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.979595] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.979758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.979879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.980119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.980365] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.980519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.980638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.980846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.980991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.981194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.981455] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.981608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.981725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.981932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.982065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  660.982297] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  660.982449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  660.982566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  660.982770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  660.982904] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  660.983133] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 427a084e8df64c37b83539b611832fe5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  660.988310] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 427a084e8df64c37b83539b611832fe5
[  661.269328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  661.269953] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  661.269953] env[60890]: ERROR oslo_messaging.rpc.server 
[  661.485333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  661.485858] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  661.485858] env[60890]: ERROR oslo_messaging.rpc.server 
[  661.843053] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  661.843254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  661.843380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  661.843647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  661.843800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  661.844086] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  661.844244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  661.844363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  661.844640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  661.844949] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  661.845109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  661.845229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  661.845442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  661.845582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  661.845728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  661.845975] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  661.846127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  661.846244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  661.846451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  661.846581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  661.846829] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-03ec354e-eb2c-413a-ad04-a12bdd597e51 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  661.846978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  661.847093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  661.847295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  661.847424] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03ec354e-eb2c-413a-ad04-a12bdd597e51',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  661.847764] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0297edccf7714b13a588b79265673d8d in queue reply_156db6c4256a4c58884afce15e6f5340
[  661.853229] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0297edccf7714b13a588b79265673d8d
[  662.028767] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  662.029067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  662.029174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  662.029360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  662.029523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  662.029754] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  662.029905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  662.030021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  662.030229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  662.030471] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  662.030710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  662.030854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  662.031070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  662.031209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  662.031358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  662.031627] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  662.031755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  662.031929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  662.032209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  662.032306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  662.033027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  662.033299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  662.033616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  662.033980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  662.034184] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  662.034584] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d41db71f7a5b4b1b999008fdce6daed1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  662.042453] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d41db71f7a5b4b1b999008fdce6daed1
[  662.350347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-03ec354e-eb2c-413a-ad04-a12bdd597e51" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  662.351034] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  662.351034] env[60890]: ERROR oslo_messaging.rpc.server 
[  662.538181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  662.538684] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  662.538684] env[60890]: ERROR oslo_messaging.rpc.server 
[  663.094819] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.094819] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.094819] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.094819] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.094819] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.094819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.094819] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  663.094819] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6c81700310e44cca989a589f3a9f99a5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  663.098676] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6c81700310e44cca989a589f3a9f99a5
[  663.270115] env[60890]: INFO networking_nsxv3.common.synchronization [req-05ecc47b-1e49-4859-90a9-ffeabd15eafe req-22fa2224-84c3-4545-8ae8-119758a326a6 tempest-ServerTagsTestJSON-644809188 tempest-ServerTagsTestJSON-644809188-project-member] Enqueued Resource ID: 8e84b2b5-833c-4fe0-b63b-f1d9445f086f with Priority: HIGHEST for action security_group_members
[  663.270115] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8e84b2b5-833c-4fe0-b63b-f1d9445f086f with Priority: HIGHEST for action security_group_members
[  663.270115] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.270115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.270115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.270115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.270115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.270115] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.270115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.271012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.271012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.271012] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.271012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.271012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.271304] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.271423] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.271556] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('8e84b2b5-833c-4fe0-b63b-f1d9445f086f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  663.271765] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4f99b7da6b0f47b3a841636ce7b1527f in queue reply_156db6c4256a4c58884afce15e6f5340
[  663.286873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4f99b7da6b0f47b3a841636ce7b1527f
[  663.595385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  663.596354] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  663.596354] env[60890]: ERROR oslo_messaging.rpc.server 
[  663.775195] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.775401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.775401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.775635] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.775736] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.775958] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  663.776115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  663.776162] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  663.776303] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  663.776424] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  663.776459] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 8e84b2b5-833c-4fe0-b63b-f1d9445f086f already deleted.
[  663.776538] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-8e84b2b5-833c-4fe0-b63b-f1d9445f086f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.241338] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.241628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.241668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.241950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.242094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.242342] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.242491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.242605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.242857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.243103] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.243251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.243372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.243572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.243708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.243889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.244237] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.244392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.244506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.244749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.244923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.245217] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.245372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.245518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.245798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.245973] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  664.246212] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 75a8968d6b494185864f7d5716a79b0d in queue reply_156db6c4256a4c58884afce15e6f5340
[  664.253272] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 75a8968d6b494185864f7d5716a79b0d
[  664.641132] env[60890]: INFO networking_nsxv3.common.synchronization [req-d6f050a3-7212-47a2-aa3e-26877b53977a req-f4c8bdeb-34a6-4226-a9b6-d9229736ae04 tempest-AttachInterfacesV270Test-570971233 tempest-AttachInterfacesV270Test-570971233-project-member] Enqueued Resource ID: 2f5d7d99-d6a0-4718-b89a-392872e34f80 with Priority: HIGHEST for action security_group_members
[  664.641652] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2f5d7d99-d6a0-4718-b89a-392872e34f80 with Priority: HIGHEST for action security_group_members
[  664.641995] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.642126] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.642190] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.642393] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.642484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.642619] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-2f5d7d99-d6a0-4718-b89a-392872e34f80 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.642771] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-2f5d7d99-d6a0-4718-b89a-392872e34f80" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.642771] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-2f5d7d99-d6a0-4718-b89a-392872e34f80" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.642967] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-2f5d7d99-d6a0-4718-b89a-392872e34f80" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.643112] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  664.643203] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  664.643264] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  664.643404] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  664.643485] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.643574] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('2f5d7d99-d6a0-4718-b89a-392872e34f80',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  664.643756] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a43aa975388a475db8d1f8fe382f52f4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  664.651756] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a43aa975388a475db8d1f8fe382f52f4
[  664.748328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  664.748892] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  664.748892] env[60890]: ERROR oslo_messaging.rpc.server 
[  665.142019] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.142261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.142388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.142676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.142835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.143101] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.143257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.143376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.143655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.143903] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.144076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.144196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.144406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.144544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.144704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.144985] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.145139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.145256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.145472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.145599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.145866] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.146019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.146137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.146434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.146572] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  665.146808] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fae7a430cd124fae8c4dc2b1f53cd4a7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  665.147879] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.147988] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.148067] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.148213] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.148294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.148460] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.148549] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.148608] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.148746] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.148839] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.148931] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 2f5d7d99-d6a0-4718-b89a-392872e34f80 already deleted.
[  665.149011] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-2f5d7d99-d6a0-4718-b89a-392872e34f80" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.229360] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fae7a430cd124fae8c4dc2b1f53cd4a7
[  665.438133] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.438423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.438474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.438730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.438944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.439233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.439398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.439522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.439750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.439998] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.440267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.440480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.440723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.440895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.441034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.441320] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.441484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.441608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.441826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.442002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.442248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  665.442408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  665.442530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  665.442747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  665.442888] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  665.443127] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bee5e0502f32496a8fa1fb71c03859b8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  665.448230] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bee5e0502f32496a8fa1fb71c03859b8
[  665.651404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  665.651911] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  665.651911] env[60890]: ERROR oslo_messaging.rpc.server 
[  665.945904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  665.946415] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  665.946415] env[60890]: ERROR oslo_messaging.rpc.server 
[  666.586373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.586647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.586770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.586969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.587123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.587377] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.587528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.587643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.587863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.588347] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.588347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.588503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.588674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.588814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.588962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.589214] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.589364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.589480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.589686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.589817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.590078] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.590201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.590318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.590523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.590744] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  666.590891] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3672988e47f841b884f4f72cb9377805 in queue reply_156db6c4256a4c58884afce15e6f5340
[  666.597315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.597540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.597700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.598033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.598981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.598981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.598981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.599699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.599983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.600331] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.600552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.600711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.601220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.601589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.601589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.602054] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.602261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.602457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.603020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.603489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  666.603641] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  666.603923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  666.604122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  666.604752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  666.604861] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  666.605169] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f5e9d53a35b740dd85c2bcac09973b09 in queue reply_156db6c4256a4c58884afce15e6f5340
[  666.606434] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3672988e47f841b884f4f72cb9377805
[  666.610587] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f5e9d53a35b740dd85c2bcac09973b09
[  667.093371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  667.093904] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  667.093904] env[60890]: ERROR oslo_messaging.rpc.server 
[  667.108081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  667.108537] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  667.108537] env[60890]: ERROR oslo_messaging.rpc.server 
[  667.675264] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.675552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.675605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.675940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.676110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.676373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-238 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.676540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.676653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.676866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.677154] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.677306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.677422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.677623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.677765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.677909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-238" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.678193] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.678296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.678454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.678702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.678837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.679072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5ea3962d-65af-41d0-a743-7d10d3b348c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.679220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.679334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.679545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.679668] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5ea3962d-65af-41d0-a743-7d10d3b348c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  667.679916] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7ba6c43c737449468dcf0ce944b26adb in queue reply_156db6c4256a4c58884afce15e6f5340
[  667.682469] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.682748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.682915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.683349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.683349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.683591] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.683779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.683956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.685233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.685233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.685233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.685233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.685477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.685669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.685857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.686921] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.687153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.687319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.687570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.687755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  667.688124] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  667.688351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  667.688655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  667.688765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  667.688937] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  667.689199] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cde2e7b9e7e04eed920dbb8c04a4386f in queue reply_156db6c4256a4c58884afce15e6f5340
[  667.690378] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7ba6c43c737449468dcf0ce944b26adb
[  667.694197] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cde2e7b9e7e04eed920dbb8c04a4386f
[  668.191423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5ea3962d-65af-41d0-a743-7d10d3b348c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  668.192050] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  668.192050] env[60890]: ERROR oslo_messaging.rpc.server 
[  668.193268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  668.193646] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  668.193646] env[60890]: ERROR oslo_messaging.rpc.server 
[  668.765333] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  668.765622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  668.765672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  668.765979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  668.766125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  668.766985] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  668.766985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  668.766985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  668.766985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  668.767181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  668.767253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  668.767367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  668.767574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  668.767722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  668.767871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  668.768146] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  668.768301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  668.768418] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  668.768625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  668.768756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  668.768992] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  668.769141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  668.769257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  668.769459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  668.769831] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  668.769831] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 97fdf9bdfeb049fe9859d9b7e0b28a59 in queue reply_156db6c4256a4c58884afce15e6f5340
[  668.775584] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 97fdf9bdfeb049fe9859d9b7e0b28a59
[  669.008937] env[60890]: INFO networking_nsxv3.common.synchronization [req-077777b7-939e-443c-9ab8-671039fc4360 req-33fd5ee1-0c54-49f1-b3e9-4d4b4c03c474 tempest-VolumesAdminNegativeTest-2086354477 tempest-VolumesAdminNegativeTest-2086354477-project-member] Enqueued Resource ID: a4acf57a-8478-4dd9-b673-474172bcc683 with Priority: HIGHEST for action security_group_members
[  669.009495] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: a4acf57a-8478-4dd9-b673-474172bcc683 with Priority: HIGHEST for action security_group_members
[  669.009823] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.009946] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.010016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.010985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.010985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.010985] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-a4acf57a-8478-4dd9-b673-474172bcc683 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.010985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-a4acf57a-8478-4dd9-b673-474172bcc683" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.010985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-a4acf57a-8478-4dd9-b673-474172bcc683" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.010985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-a4acf57a-8478-4dd9-b673-474172bcc683" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.010985] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.010985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.010985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.011502] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.011502] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.011502] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('a4acf57a-8478-4dd9-b673-474172bcc683',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  669.011502] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6755edfe0d204f328a4bfe945d29e40c in queue reply_156db6c4256a4c58884afce15e6f5340
[  669.025212] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6755edfe0d204f328a4bfe945d29e40c
[  669.271902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  669.272521] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  669.272521] env[60890]: ERROR oslo_messaging.rpc.server 
[  669.513974] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.514144] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.514227] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.514436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.514527] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.514780] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.514934] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.515005] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.515157] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.515234] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.515319] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: a4acf57a-8478-4dd9-b673-474172bcc683 already deleted.
[  669.515394] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-a4acf57a-8478-4dd9-b673-474172bcc683" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.862726] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.863047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.863047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.863328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.863476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.863734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.863895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.864035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.864254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.864498] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.864654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.864832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.865049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.865193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.865343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.865593] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.865768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.865956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.866179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.866320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  669.866561] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  669.866719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  669.866844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  669.868081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  669.868081] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  669.868081] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 845f06df21954c36865d05fd46546715 in queue reply_156db6c4256a4c58884afce15e6f5340
[  669.872636] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 845f06df21954c36865d05fd46546715
[  670.321014] env[60890]: INFO networking_nsxv3.common.synchronization [req-b2333001-089d-44cd-82da-383f814287ca req-a54e481c-556d-4faf-a925-44385d488cac tempest-ServerActionsTestOtherA-363397817 tempest-ServerActionsTestOtherA-363397817-project-member] Enqueued Resource ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f with Priority: HIGHEST for action security_group_members
[  670.321014] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f with Priority: HIGHEST for action security_group_members
[  670.321014] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.321014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.321014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.321014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.321014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.321014] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f934cb7a-eebb-4560-bd27-6e115d57ba4f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.321014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.321014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.324455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.324723] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.324890] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.325006] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.325221] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.327017] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.327193] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f934cb7a-eebb-4560-bd27-6e115d57ba4f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  670.327444] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1805d10b4bd04d0e99769bb87a250c81 in queue reply_156db6c4256a4c58884afce15e6f5340
[  670.342433] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1805d10b4bd04d0e99769bb87a250c81
[  670.369684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  670.370203] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  670.370203] env[60890]: ERROR oslo_messaging.rpc.server 
[  670.635857] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.636082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.636241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.636494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.636641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.636970] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.637095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.641680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.641680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.641680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.641680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.641680] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  670.641680] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b7f56e067ca5421fac56b0564fb0773e in queue reply_156db6c4256a4c58884afce15e6f5340
[  670.646202] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b7f56e067ca5421fac56b0564fb0773e
[  670.831929] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.832103] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.832176] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.832772] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.832772] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.832772] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.832772] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.833026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.833026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.833106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.833138] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f already deleted.
[  670.833204] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.914960] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.915259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.915301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.915599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.915785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.916075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.916244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.916363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.916592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.916980] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.917082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.917158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.917366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.917515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.917664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.917918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.918072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.918191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.918472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.918607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  670.918892] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  670.919916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  670.919916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  670.919916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  670.919916] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  670.919916] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 30331d2bc815428582a28804623d6909 in queue reply_156db6c4256a4c58884afce15e6f5340
[  670.926944] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 30331d2bc815428582a28804623d6909
[  671.145704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  671.146260] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  671.146260] env[60890]: ERROR oslo_messaging.rpc.server 
[  671.423468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  671.424034] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  671.424034] env[60890]: ERROR oslo_messaging.rpc.server 
[  671.725708] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  671.725916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  671.726051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  671.726329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  671.726482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  671.726753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  671.726916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  671.727055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  671.727303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  671.727557] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  671.727717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  671.727839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  671.728071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  671.728221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  671.728376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  671.728628] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  671.728787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  671.728910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  671.729122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  671.729260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  671.729502] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  671.729660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  671.729780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  671.729991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  671.730127] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  671.730352] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8cf495d3ac484af1b870cb8c00f1972b in queue reply_156db6c4256a4c58884afce15e6f5340
[  671.735957] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8cf495d3ac484af1b870cb8c00f1972b
[  672.232899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  672.233408] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  672.233408] env[60890]: ERROR oslo_messaging.rpc.server 
[  672.314716] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.315004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.315188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.315558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.315775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.316165] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.316388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.316559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.316860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.317214] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.317768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.317768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.317889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.318071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.318290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.318658] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.318903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.319081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.319374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.319570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.319918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.320984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.321183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.321482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.321688] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  672.322008] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bbbef47cf9744b70978d16a391dc63a5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  672.328491] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bbbef47cf9744b70978d16a391dc63a5
[  672.827278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  672.827278] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  672.827278] env[60890]: ERROR oslo_messaging.rpc.server 
[  672.916656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.916656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.916656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.916656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  672.916656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  672.916656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  672.916656] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  672.916656] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 37cca0f26b294130b1004f334d5a07d8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  672.922324] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 37cca0f26b294130b1004f334d5a07d8
[  673.187401] env[60890]: INFO networking_nsxv3.common.synchronization [req-1669e5d3-ce16-40b5-af7e-e6a35bf5611b req-931a5d82-fd57-4472-b617-8e8c743e0590 tempest-AttachVolumeShelveTestJSON-2132759461 tempest-AttachVolumeShelveTestJSON-2132759461-project-member] Enqueued Resource ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 with Priority: HIGHEST for action security_group_members
[  673.187911] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 with Priority: HIGHEST for action security_group_members
[  673.188237] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.188358] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.188424] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.188611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.188689] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.188838] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.188943] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.188989] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.189160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.189296] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.189388] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.189435] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.189569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.189644] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.189725] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f80ac04b-9522-404c-9dc2-d2e2b8a228d9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  673.189899] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2468fd214fff4d95b7ce2ed2c71ccda8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  673.208022] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2468fd214fff4d95b7ce2ed2c71ccda8
[  673.379248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.379502] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.379583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.379851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.380010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.380296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.380454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.380572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.380798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.381041] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.381194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.381313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.381519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.388140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.388140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.388140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.388140] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  673.388140] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6259ea13ed564b9a9d7bd49e02463074 in queue reply_156db6c4256a4c58884afce15e6f5340
[  673.393196] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6259ea13ed564b9a9d7bd49e02463074
[  673.421922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  673.422682] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  673.422682] env[60890]: ERROR oslo_messaging.rpc.server 
[  673.693265] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.693422] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.693481] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.693868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.693868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.694017] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.694113] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.694177] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.694368] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.694408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.694514] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 already deleted.
[  673.694556] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.789997] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.790194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.790322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.790878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.790878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.790996] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.791192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.791259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.791531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.791781] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.791954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.792072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.792293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.792440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.792595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.792944] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.793107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.793230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.793442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.793580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.793819] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  673.793977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  673.794096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  673.794340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  673.794479] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  673.794778] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c5d76baefe014595b52d97614b3559f9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  673.804252] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c5d76baefe014595b52d97614b3559f9
[  673.885421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  673.885984] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  673.885984] env[60890]: ERROR oslo_messaging.rpc.server 
[  674.014220] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.014432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.014562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.014880] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.015039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.015301] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.015462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.015583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.015806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.016066] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.016228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.016347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.016638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.016703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.016845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.017121] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.017279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.017431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.017607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.017746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.018050] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.018209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.018328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.018537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.018673] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  674.018908] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg efc110c0b9634f65a5cefbd9b93388f1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  674.024300] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg efc110c0b9634f65a5cefbd9b93388f1
[  674.297255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  674.297804] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  674.297804] env[60890]: ERROR oslo_messaging.rpc.server 
[  674.504231] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.504532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.504589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.504896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.505043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.505298] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.505454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.505568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.505790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.506072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.506229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.506345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.506552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.506692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.506836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.507093] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.507245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.507367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.507594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.507736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.507975] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.508136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.508251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.508762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.508762] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  674.508869] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1e3b683b533f48a9b82e6dec7f0c4b1e in queue reply_156db6c4256a4c58884afce15e6f5340
[  674.513867] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1e3b683b533f48a9b82e6dec7f0c4b1e
[  674.521149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  674.521520] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  674.521520] env[60890]: ERROR oslo_messaging.rpc.server 
[  674.860278] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.860278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.860278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.860278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.860278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.860783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.860783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.860783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.860978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.864032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.864032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  674.864032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  674.864032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  674.864032] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  674.864032] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f2577b11a6844755b1e95f24a235cec2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  674.869164] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f2577b11a6844755b1e95f24a235cec2
[  675.010898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  675.011404] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  675.011404] env[60890]: ERROR oslo_messaging.rpc.server 
[  675.140767] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.140979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.141102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.141361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.141503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.141764] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.141922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.142056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.142245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.142489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.142673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.142811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.143029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.143175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.143324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.143578] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.143735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.143856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.144087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.144229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.144468] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.144614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.144790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.145022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.145158] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  675.145740] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 939534ca31ea4799b9d834a37edc435f in queue reply_156db6c4256a4c58884afce15e6f5340
[  675.182720] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 939534ca31ea4799b9d834a37edc435f
[  675.366270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  675.366785] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  675.366785] env[60890]: ERROR oslo_messaging.rpc.server 
[  675.577160] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.577473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.577473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.577741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.577894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.578157] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.578309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.578423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.578640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.578897] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.579048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.579165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.579368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.579507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.579651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.579896] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.580080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.580205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.580409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.580540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.580777] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d227608b-b9af-4f25-a933-8b75375430a2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.580931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.581047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.581721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.581920] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d227608b-b9af-4f25-a933-8b75375430a2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  675.582168] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c78511a6c04e4c8d8195f99acfb1a816 in queue reply_156db6c4256a4c58884afce15e6f5340
[  675.588253] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c78511a6c04e4c8d8195f99acfb1a816
[  675.648792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  675.649300] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  675.649300] env[60890]: ERROR oslo_messaging.rpc.server 
[  675.824628] env[60890]: INFO networking_nsxv3.common.synchronization [None req-846772d1-14bd-41bf-8295-93c6e8593e87 None None] Enqueued Resource ID: c9b46da7-94d4-40f7-925e-c927ee5ee606 with Priority: HIGHEST for action port
[  675.824934] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c9b46da7-94d4-40f7-925e-c927ee5ee606 with Priority: HIGHEST for action port
[  675.825520] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.825645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.825713] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.825922] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.826003] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  675.826155] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-c9b46da7-94d4-40f7-925e-c927ee5ee606 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  675.826246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-c9b46da7-94d4-40f7-925e-c927ee5ee606" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  675.826305] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-c9b46da7-94d4-40f7-925e-c927ee5ee606" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  675.826499] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-c9b46da7-94d4-40f7-925e-c927ee5ee606" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  675.826575] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c9b46da7-94d4-40f7-925e-c927ee5ee606',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  675.827131] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8d146357f1584a9e8c4ae95cae9280c7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  675.837669] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8d146357f1584a9e8c4ae95cae9280c7
[  676.038948] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.039158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.039292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.039560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.039711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.039973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.040149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.040361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.040484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.040734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.040914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.041047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.041255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.041396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.041548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.042244] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.042244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.042244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.042456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.042594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.042836] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.042991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.043109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.043316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.043452] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  676.043687] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e43c3adcc7a446bda437df7b6712c6a5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  676.052053] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e43c3adcc7a446bda437df7b6712c6a5
[  676.084599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d227608b-b9af-4f25-a933-8b75375430a2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  676.085196] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  676.085196] env[60890]: ERROR oslo_messaging.rpc.server 
[  676.203783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.203783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.203783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.203783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.203783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.203783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.206585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.206585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.206585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.206585] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  676.206585] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3688b98feebc40d3a093d3a6d0437937 in queue reply_156db6c4256a4c58884afce15e6f5340
[  676.211523] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3688b98feebc40d3a093d3a6d0437937
[  676.329597] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-c9b46da7-94d4-40f7-925e-c927ee5ee606" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.329851] env[60890]: Traceback (most recent call last):
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  676.329920] env[60890]:     timer()
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  676.329920] env[60890]:     cb(*args, **kw)
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  676.329920] env[60890]:     result = function(*args, **kwargs)
[  676.329920] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  676.329920] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  676.329920] env[60890]:     return method(*args, **kwargs)
[  676.329920] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  676.329920] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  676.329920] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  676.329920] env[60890]:     result = self.transport._send(
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  676.329920] env[60890]:     return self._driver.send(target, ctxt, message,
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  676.329920] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  676.329920] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  676.329920] env[60890]:     raise result
[  676.329920] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  676.329920] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  676.546824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  676.547477] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  676.547477] env[60890]: ERROR oslo_messaging.rpc.server 
[  676.707985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  676.708554] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  676.708554] env[60890]: ERROR oslo_messaging.rpc.server 
[  676.901864] env[60890]: INFO networking_nsxv3.common.synchronization [req-d6f050a3-7212-47a2-aa3e-26877b53977a req-df21096a-b8ed-4bc0-a395-a17c6226d75f tempest-AttachInterfacesV270Test-570971233 tempest-AttachInterfacesV270Test-570971233-project-member] Enqueued Resource ID: 2f5d7d99-d6a0-4718-b89a-392872e34f80 with Priority: HIGHEST for action security_group_members
[  676.901864] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2f5d7d99-d6a0-4718-b89a-392872e34f80 with Priority: HIGHEST for action security_group_members
[  676.901864] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.901864] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-2f5d7d99-d6a0-4718-b89a-392872e34f80 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-2f5d7d99-d6a0-4718-b89a-392872e34f80" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-2f5d7d99-d6a0-4718-b89a-392872e34f80" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-2f5d7d99-d6a0-4718-b89a-392872e34f80" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.901864] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  676.901864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  676.901864] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('2f5d7d99-d6a0-4718-b89a-392872e34f80',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  676.902705] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e67cfd3693c245278a5564bd510674d5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  676.916597] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e67cfd3693c245278a5564bd510674d5
[  677.198975] env[60890]: INFO networking_nsxv3.common.synchronization [req-904c238b-5702-4769-898d-0e206101cbfb req-fc4cc56f-047f-4b2f-8bbb-7ee500d06724 tempest-ServerAddressesTestJSON-521927788 tempest-ServerAddressesTestJSON-521927788-project-member] Enqueued Resource ID: 458f359e-9b52-473c-8206-6b4996c8c6d7 with Priority: HIGHEST for action security_group_members
[  677.198975] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 458f359e-9b52-473c-8206-6b4996c8c6d7 with Priority: HIGHEST for action security_group_members
[  677.198975] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.198975] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-458f359e-9b52-473c-8206-6b4996c8c6d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-458f359e-9b52-473c-8206-6b4996c8c6d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-458f359e-9b52-473c-8206-6b4996c8c6d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-458f359e-9b52-473c-8206-6b4996c8c6d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.198975] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.198975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.198975] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('458f359e-9b52-473c-8206-6b4996c8c6d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  677.198975] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d84901dfcc7b4abaa921774cae1ae3f4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  677.210480] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d84901dfcc7b4abaa921774cae1ae3f4
[  677.327753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.327753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.327753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.327753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.327753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.333095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.333284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.333429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.333762] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.333947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.334070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.334274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.334418] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  677.334653] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg de969361722b4f5ca87845aa303dc7dc in queue reply_156db6c4256a4c58884afce15e6f5340
[  677.341597] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg de969361722b4f5ca87845aa303dc7dc
[  677.404576] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.404820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.404913] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.405127] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.405227] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.405417] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.405520] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.405584] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.405790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.405883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.405973] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 2f5d7d99-d6a0-4718-b89a-392872e34f80 already deleted.
[  677.406056] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-2f5d7d99-d6a0-4718-b89a-392872e34f80" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.507370] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.507576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.507706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.508022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.508159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.508427] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.508606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.508733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.508955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.509208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.509375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.509499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.509711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.509868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.510020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.510282] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.510440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.510560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.510772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.510913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.511157] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.511343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.511424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.511672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.511767] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  677.512016] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 30e35101dd5c4ed8bf630ffb3c9ddc5d in queue reply_156db6c4256a4c58884afce15e6f5340
[  677.522318] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 30e35101dd5c4ed8bf630ffb3c9ddc5d
[  677.634323] env[60890]: INFO networking_nsxv3.common.synchronization [None req-2f26a63e-3bbb-4299-823a-8e07dc3511a1 tempest-ServersTestFqdnHostnames-535084876 tempest-ServersTestFqdnHostnames-535084876-project-member] Enqueued Resource ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b with Priority: HIGHEST for action security_group_rules
[  677.634886] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b with Priority: HIGHEST for action security_group_rules
[  677.635202] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.635326] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.635394] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.635594] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.635738] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.635856] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-337aaaf2-15f5-46a8-8b59-236cfe63466b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.635957] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.636053] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.636266] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.636345] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('337aaaf2-15f5-46a8-8b59-236cfe63466b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  677.636530] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f3c2c26ee4f647479f463ef3ef1e007f in queue reply_156db6c4256a4c58884afce15e6f5340
[  677.651671] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f3c2c26ee4f647479f463ef3ef1e007f
[  677.701035] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.701143] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.701197] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.701415] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.701503] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.701934] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.701934] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.701934] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.702120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.702120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.702193] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 458f359e-9b52-473c-8206-6b4996c8c6d7 already deleted.
[  677.702240] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-458f359e-9b52-473c-8206-6b4996c8c6d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.710636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.710942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.711136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.711206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.711377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.711594] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.711760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.711860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.712549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.712549] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.712696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.712757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.712967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.713174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.713248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.713491] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.713640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.713766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.713993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.714134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.714372] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.714521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.714637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.715184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.715184] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  677.715300] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a4985d8211c34841a4e46b0f5475d40b in queue reply_156db6c4256a4c58884afce15e6f5340
[  677.722496] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a4985d8211c34841a4e46b0f5475d40b
[  677.757771] env[60890]: INFO networking_nsxv3.common.synchronization [None req-f60a1ae1-113f-4239-8630-2c713eb55b96 tempest-ServersTestFqdnHostnames-535084876 tempest-ServersTestFqdnHostnames-535084876-project-member] Enqueued Resource ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b with Priority: HIGHEST for action security_group_rules
[  677.758260] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b with Priority: HIGHEST for action security_group_rules
[  677.758570] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.758692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.758760] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.758952] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.759039] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.759186] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-337aaaf2-15f5-46a8-8b59-236cfe63466b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.759261] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.829956] env[60890]: INFO networking_nsxv3.common.synchronization [None req-d9a9c5e5-9e18-435b-87d4-b9b8d5d0c3f1 None None] Enqueued Resource ID: 74dba47b-b4e6-43f3-ae5e-f4088a6a2701 with Priority: HIGHEST for action port
[  677.830392] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 74dba47b-b4e6-43f3-ae5e-f4088a6a2701 with Priority: HIGHEST for action port
[  677.830754] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.830876] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.830941] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.831142] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.831226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.831372] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-74dba47b-b4e6-43f3-ae5e-f4088a6a2701 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  677.831460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-74dba47b-b4e6-43f3-ae5e-f4088a6a2701" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  677.831513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-74dba47b-b4e6-43f3-ae5e-f4088a6a2701" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  677.832259] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-74dba47b-b4e6-43f3-ae5e-f4088a6a2701" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  677.832259] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('74dba47b-b4e6-43f3-ae5e-f4088a6a2701',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  677.832259] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 42412733a5124e519e31cbf9c1d45862 in queue reply_156db6c4256a4c58884afce15e6f5340
[  677.837416] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 42412733a5124e519e31cbf9c1d45862
[  677.837687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  677.838141] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  677.838141] env[60890]: ERROR oslo_messaging.rpc.server 
[  678.018055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  678.018560] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  678.018560] env[60890]: ERROR oslo_messaging.rpc.server 
[  678.162359] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.162517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.162624] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.162899] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.162998] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.163094] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b already deleted.
[  678.163186] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.163362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.163462] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.163541] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('337aaaf2-15f5-46a8-8b59-236cfe63466b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  678.163727] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b950c827bcef4ab1a0442ff510835544 in queue reply_156db6c4256a4c58884afce15e6f5340
[  678.181249] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b950c827bcef4ab1a0442ff510835544
[  678.217798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  678.218333] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  678.218333] env[60890]: ERROR oslo_messaging.rpc.server 
[  678.334611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-74dba47b-b4e6-43f3-ae5e-f4088a6a2701" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.334915] env[60890]: Traceback (most recent call last):
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  678.334987] env[60890]:     timer()
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  678.334987] env[60890]:     cb(*args, **kw)
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  678.334987] env[60890]:     result = function(*args, **kwargs)
[  678.334987] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  678.334987] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  678.334987] env[60890]:     return method(*args, **kwargs)
[  678.334987] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  678.334987] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  678.334987] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  678.334987] env[60890]:     result = self.transport._send(
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  678.334987] env[60890]:     return self._driver.send(target, ctxt, message,
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  678.334987] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  678.334987] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  678.334987] env[60890]:     raise result
[  678.334987] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  678.334987] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  678.380497] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.380706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.380839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.381121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.381292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.381559] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.381719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.381842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.382066] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.382315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.382473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.382594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.382842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.382996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.383149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.383404] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.383560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.383680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.383932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.384123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.384373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.384531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.384652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.384907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.385047] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  678.385278] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4b8515a6643e42c1b997f466c7c5c62d in queue reply_156db6c4256a4c58884afce15e6f5340
[  678.390788] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4b8515a6643e42c1b997f466c7c5c62d
[  678.638784] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.638784] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.638784] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.638784] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.638784] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.638784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.638784] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  678.638784] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b40f0f55cac24494873e5bbc60dd7c00 in queue reply_156db6c4256a4c58884afce15e6f5340
[  678.646684] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b40f0f55cac24494873e5bbc60dd7c00
[  678.688747] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.688949] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.689047] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.689332] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.689478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.689585] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b already deleted.
[  678.689710] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.822885] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.823212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.823313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.823592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.823746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.824281] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.824281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.824411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.824558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.825231] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.825427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.825514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.825800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.826641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.826641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.826641] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.826641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.826641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.827185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.827185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.827313] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  678.827683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  678.827683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  678.827803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  678.827892] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  678.828143] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg acf0ec030ace477aa99727d9a4f0940e in queue reply_156db6c4256a4c58884afce15e6f5340
[  678.834819] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg acf0ec030ace477aa99727d9a4f0940e
[  678.887898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  678.888457] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  678.888457] env[60890]: ERROR oslo_messaging.rpc.server 
[  679.140861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  679.141386] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  679.141386] env[60890]: ERROR oslo_messaging.rpc.server 
[  679.330905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  679.331434] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  679.331434] env[60890]: ERROR oslo_messaging.rpc.server 
[  679.524121] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.524439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.524512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.524795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.524966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.525234] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.525480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.525612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.525844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.526080] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.526239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.526362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.526572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.526719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.526872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.527160] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.527332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.527455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.527668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.527807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.528086] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.528219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.528337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.528541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.528675] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  679.528955] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a6192069cd414b8c970d10838b488eea in queue reply_156db6c4256a4c58884afce15e6f5340
[  679.536146] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a6192069cd414b8c970d10838b488eea
[  679.682773] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.682994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.683164] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.683444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.683596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.683854] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.684025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.684195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.684429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.684789] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.684975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.685097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.685451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.685657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.685910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.686248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.686453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.686643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.686934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.687094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.687362] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.687491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.687609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.687822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.687959] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  679.688212] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 40ff336e16e74f86b255c468bd66b763 in queue reply_156db6c4256a4c58884afce15e6f5340
[  679.694994] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 40ff336e16e74f86b255c468bd66b763
[  679.720092] env[60890]: INFO networking_nsxv3.common.synchronization [req-e9cf7b58-3f2b-435a-8967-4678520957f4 req-0bb3371a-366d-44a9-8bc5-6e56ac4ff38b tempest-ServerMetadataTestJSON-491230402 tempest-ServerMetadataTestJSON-491230402-project-member] Enqueued Resource ID: 617466de-da46-4007-bf41-f1add494a4d0 with Priority: HIGHEST for action security_group_members
[  679.720389] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 617466de-da46-4007-bf41-f1add494a4d0 with Priority: HIGHEST for action security_group_members
[  679.720947] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.721099] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.721235] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.721467] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.721568] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.721764] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-617466de-da46-4007-bf41-f1add494a4d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.721881] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-617466de-da46-4007-bf41-f1add494a4d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.722106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-617466de-da46-4007-bf41-f1add494a4d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.722209] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-617466de-da46-4007-bf41-f1add494a4d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.722403] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.722514] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.722572] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.727492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.727492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.727492] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('617466de-da46-4007-bf41-f1add494a4d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  679.727492] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 95f5993eca6b4285a2a7cdf5bae2cc1a in queue reply_156db6c4256a4c58884afce15e6f5340
[  679.737954] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 95f5993eca6b4285a2a7cdf5bae2cc1a
[  679.933051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.933375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.933375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.933623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.933766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.934019] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.934173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.934289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.934515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.934779] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.934944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.935122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.935354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.935519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.935694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.935979] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.936148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.936291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.936517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.936653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  679.936890] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  679.937038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  679.937154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  679.937638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  679.937785] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  679.938019] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c44d088610f4401594a3712aaee5e530 in queue reply_156db6c4256a4c58884afce15e6f5340
[  679.943207] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c44d088610f4401594a3712aaee5e530
[  680.031627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  680.032149] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  680.032149] env[60890]: ERROR oslo_messaging.rpc.server 
[  680.190563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  680.191179] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  680.191179] env[60890]: ERROR oslo_messaging.rpc.server 
[  680.225646] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.225891] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.225957] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.226177] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.226279] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.226480] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.226607] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.226703] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.226883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.226965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.227052] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 617466de-da46-4007-bf41-f1add494a4d0 already deleted.
[  680.227134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-617466de-da46-4007-bf41-f1add494a4d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.317050] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.317283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.317420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.317695] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.317846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.318117] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.318271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.318386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.318713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.318983] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.319138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.319256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.319463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.319604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.319836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.320011] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.320610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.320770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.321038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.321208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.322718] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.322718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.322718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.322718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.322718] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  680.322718] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 11d66efcb9bc47359a4fd3cd406878d8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  680.329138] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 11d66efcb9bc47359a4fd3cd406878d8
[  680.439893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  680.440490] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  680.440490] env[60890]: ERROR oslo_messaging.rpc.server 
[  680.804068] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.804297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.804428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.804773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.804944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.805230] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-704 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.805406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.805528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.805764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.806008] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.806166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.806286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.806496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.806641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.806789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-704" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.807043] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.807196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.807317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.807527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.807663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.807901] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-9832c8d7-e9c9-4644-b91f-78a843c1824c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.808081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.808210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.808442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.808668] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9832c8d7-e9c9-4644-b91f-78a843c1824c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  680.808913] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg aaeb06eac7db491abeb1484057854ecc in queue reply_156db6c4256a4c58884afce15e6f5340
[  680.815596] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg aaeb06eac7db491abeb1484057854ecc
[  680.825082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  680.825524] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  680.825524] env[60890]: ERROR oslo_messaging.rpc.server 
[  680.883014] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.883271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.883341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.883616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.883764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.884027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.884194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.884396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.884627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.884947] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.885110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.885233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.885445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.885593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.885827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.886201] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.886353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.886353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.886670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.886670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.886911] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.887074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.887184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.887380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.887502] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  680.887740] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 044b7a90ba62466b92aebdcec827fb39 in queue reply_156db6c4256a4c58884afce15e6f5340
[  680.894576] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 044b7a90ba62466b92aebdcec827fb39
[  680.980553] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.980886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.980886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.981163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.981312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.981578] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.981739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.981859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.982079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.982326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.982486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.982609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.982818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.982966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.983119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.983371] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.983527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.983648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.983860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.983998] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  680.984260] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  680.984417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  680.984539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  680.984806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  680.984960] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  680.985197] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 97321bdfe2394c8aa5009b34f516c6f3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  680.990419] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 97321bdfe2394c8aa5009b34f516c6f3
[  681.311649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-9832c8d7-e9c9-4644-b91f-78a843c1824c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  681.312162] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  681.312162] env[60890]: ERROR oslo_messaging.rpc.server 
[  681.377589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.377802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.377931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.378210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.378360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.378621] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.378782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.378903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.379126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.379373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.379532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.379655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.379863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.380016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.380171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.380481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.380563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.380738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.380884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.381023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.381264] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.381422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.381541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.381752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.381892] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  681.382126] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0e57d2b72f2a42c8b760f6462ebed2c8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  681.387200] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0e57d2b72f2a42c8b760f6462ebed2c8
[  681.389950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  681.390337] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  681.390337] env[60890]: ERROR oslo_messaging.rpc.server 
[  681.487324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  681.487908] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  681.487908] env[60890]: ERROR oslo_messaging.rpc.server 
[  681.827343] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.827343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.827473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.828252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.828428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.828698] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.828858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.829003] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.829232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.829478] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.829631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.829749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.829958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.830098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.830308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.830500] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.830651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.830778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.830985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.831118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.831353] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  681.831505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  681.831623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  681.831826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  681.831960] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  681.832943] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6dc6b555f6554ddea58642c012e6d268 in queue reply_156db6c4256a4c58884afce15e6f5340
[  681.856830] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6dc6b555f6554ddea58642c012e6d268
[  681.884325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  681.885225] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  681.885225] env[60890]: ERROR oslo_messaging.rpc.server 
[  682.188130] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.188404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.188404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.188598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.188750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.189006] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.189165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.189287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.189520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.189769] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.189925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.190043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.190247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.190390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.190536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.190785] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.190934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.191049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.191273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.191384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.191653] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.191810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.191882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.192103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.192233] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  682.192468] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3eeeaccd9eb149aab6f9fb3d23143ac0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  682.198550] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3eeeaccd9eb149aab6f9fb3d23143ac0
[  682.335474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  682.336150] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  682.336150] env[60890]: ERROR oslo_messaging.rpc.server 
[  682.352869] env[60890]: INFO networking_nsxv3.common.synchronization [req-b2333001-089d-44cd-82da-383f814287ca req-5af5850b-b965-493a-a884-b50bd478764b tempest-ServerActionsTestOtherA-363397817 tempest-ServerActionsTestOtherA-363397817-project-member] Enqueued Resource ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f with Priority: HIGHEST for action security_group_members
[  682.353436] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f with Priority: HIGHEST for action security_group_members
[  682.353809] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.353958] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.354220] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.354275] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.354323] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.354498] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f934cb7a-eebb-4560-bd27-6e115d57ba4f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.354596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.354657] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.354945] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.355089] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.355184] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.355243] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.355386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.355462] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.355590] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f934cb7a-eebb-4560-bd27-6e115d57ba4f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  682.355883] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4b8e804e9cda446e8ebe8dc807fd6c4b in queue reply_156db6c4256a4c58884afce15e6f5340
[  682.363489] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4b8e804e9cda446e8ebe8dc807fd6c4b
[  682.472045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.472045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.472045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.472045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.472045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.472045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.472045] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  682.472045] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d86d38ea171241ae8b5bf4c4e7cbd73a in queue reply_156db6c4256a4c58884afce15e6f5340
[  682.477398] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d86d38ea171241ae8b5bf4c4e7cbd73a
[  682.697819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  682.697819] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  682.697819] env[60890]: ERROR oslo_messaging.rpc.server 
[  682.858650] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.858650] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.858650] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f934cb7a-eebb-4560-bd27-6e115d57ba4f already deleted.
[  682.858650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f934cb7a-eebb-4560-bd27-6e115d57ba4f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.975544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  682.976218] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  682.976218] env[60890]: ERROR oslo_messaging.rpc.server 
[  683.026615] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.026615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.026615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.026615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.026615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.026615] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.026615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.026615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.026935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.026997] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.027148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.027264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.027470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.027610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.027778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.028064] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.028209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.028325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.028541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.028670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.028904] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.029111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.029177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.029366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.029494] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  683.029720] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f2b5804d76544e399f135e364d5f2c45 in queue reply_156db6c4256a4c58884afce15e6f5340
[  683.038734] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f2b5804d76544e399f135e364d5f2c45
[  683.242332] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.242332] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.242332] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.242332] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.242332] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.242332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.242332] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  683.242332] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3ec0ee4de2714fa1872efd3aefaa32c5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  683.247070] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3ec0ee4de2714fa1872efd3aefaa32c5
[  683.512925] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.513133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.513265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.513537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.513686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.513949] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.514107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.514226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.514439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.514684] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.514945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.515089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.515300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.515441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.515589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.515897] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.516085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.516209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.516422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.516557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.516798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  683.517022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  683.517080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  683.517263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  683.517399] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  683.517636] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 34409cf52f514168a3efcc67d4ae8858 in queue reply_156db6c4256a4c58884afce15e6f5340
[  683.524185] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 34409cf52f514168a3efcc67d4ae8858
[  683.531954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  683.532407] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  683.532407] env[60890]: ERROR oslo_messaging.rpc.server 
[  683.743518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  683.744024] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  683.744024] env[60890]: ERROR oslo_messaging.rpc.server 
[  684.020491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  684.021009] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  684.021009] env[60890]: ERROR oslo_messaging.rpc.server 
[  684.178571] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.178717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.178848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.179115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.179266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.180331] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.180331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.180331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.180331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.180331] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.180551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.180551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.180707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.180848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.181037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.181241] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.181406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.181528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.181739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.181891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.182176] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.182257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.182365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.182575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.182717] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('95b3dc4f-8177-4c9d-84d0-4300d15e5e27',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  684.182972] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4987e44fbd804c6cbd156f4c599c8289 in queue reply_156db6c4256a4c58884afce15e6f5340
[  684.188459] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4987e44fbd804c6cbd156f4c599c8289
[  684.273006] env[60890]: INFO networking_nsxv3.common.synchronization [req-3c66067b-6b9c-4c02-a034-e5f2c1a1ae1c req-8555976d-fafd-4502-87d1-d48f66bfb458 tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  684.273527] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  684.273841] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.273971] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.274033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.274247] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.274339] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.274492] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.274585] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.274654] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.274837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.274970] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.275052] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.275117] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.275244] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.275406] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.275406] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  684.275572] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4c8058de33b64e45929cc2711f490a48 in queue reply_156db6c4256a4c58884afce15e6f5340
[  684.294834] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4c8058de33b64e45929cc2711f490a48
[  684.301413] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.301665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.301747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.302040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.302205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.302467] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.302625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.302744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.302970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.303427] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.303427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.303570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.303684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.303817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.303965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.304359] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.304472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.304520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.304855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.304925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.306452] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.306712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.307762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.307762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.307762] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  684.307762] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e32619abf9b241b0808df13c36adaf2f in queue reply_156db6c4256a4c58884afce15e6f5340
[  684.313558] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e32619abf9b241b0808df13c36adaf2f
[  684.456062] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0d63dbff0ba740c484e3343c52acac2f in queue reply_156db6c4256a4c58884afce15e6f5340
[  684.469813] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0d63dbff0ba740c484e3343c52acac2f
[  684.645636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.645848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.646029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.646252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.646393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.646653] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.646801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.646916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.647150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.647399] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.647549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.647661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.648070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.648070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.648246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.648422] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.648567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.648680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.648921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.649065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.649304] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.649449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.649573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.649762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.649885] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  684.650107] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 114d9653a3de4a8fa03e40d169c9c871 in queue reply_156db6c4256a4c58884afce15e6f5340
[  684.655597] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 114d9653a3de4a8fa03e40d169c9c871
[  684.686749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-95b3dc4f-8177-4c9d-84d0-4300d15e5e27" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  684.687292] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  684.687292] env[60890]: ERROR oslo_messaging.rpc.server 
[  684.779616] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.779785] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.779859] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.780103] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.780199] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.780388] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.780483] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.780544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.780698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.780772] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.780854] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  684.780931] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.809796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  684.810376] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  684.810376] env[60890]: ERROR oslo_messaging.rpc.server 
[  684.820243] env[60890]: INFO networking_nsxv3.common.synchronization [req-3c66067b-6b9c-4c02-a034-e5f2c1a1ae1c req-f2ddf7ed-0b21-43f7-a328-5409ab63d80f tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  684.820844] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  684.821176] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.821301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.821365] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.821557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.821724] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.821781] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.821868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.821925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.822065] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.822478] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  684.822478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  684.822478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  684.822478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  684.822747] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  684.822747] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  684.822826] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg fc7d117822774088aa4029a32e59bad5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  684.831187] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fc7d117822774088aa4029a32e59bad5
[  685.152053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  685.153336] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  685.153336] env[60890]: ERROR oslo_messaging.rpc.server 
[  685.324495] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.324805] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.324805] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.324884] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.324982] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.325179] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.325278] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.325334] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.325487] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.325559] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.325646] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  685.325780] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.347072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.347294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.347434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.347685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.347826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.348094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.348260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.348364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.348573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.348813] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.349053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.349114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.349288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.349645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.349645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.349821] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.350123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.350123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.350286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.350419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.350657] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.350853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.350916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.351113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.351245] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  685.351467] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b694895b4b3f4207a838487bbdc830cb in queue reply_156db6c4256a4c58884afce15e6f5340
[  685.356666] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b694895b4b3f4207a838487bbdc830cb
[  685.573551] env[60890]: INFO networking_nsxv3.common.synchronization [req-3c66067b-6b9c-4c02-a034-e5f2c1a1ae1c req-d7a609e2-b77b-4b97-b79c-e460aba33658 tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  685.574082] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  685.574399] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.574524] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.574594] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.574839] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.574929] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.575083] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.575225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.575259] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.575389] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.575528] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.575622] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.575685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.575868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.575953] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.576074] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  685.576253] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1b83ad3c5f7549b28c325c202e82d506 in queue reply_156db6c4256a4c58884afce15e6f5340
[  685.594827] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1b83ad3c5f7549b28c325c202e82d506
[  685.699544] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.699753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.699883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.700181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.700340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.700601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.700756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.700907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.701136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.701383] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.701542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.701657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.701870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.702015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.702165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.702412] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.702912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.703071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.703290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.703429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.703671] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.703907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.704103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.704326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.704467] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  685.704735] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6f0c8d82ebb54277ba33ef29acce2070 in queue reply_156db6c4256a4c58884afce15e6f5340
[  685.710283] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6f0c8d82ebb54277ba33ef29acce2070
[  685.825078] env[60890]: INFO networking_nsxv3.common.synchronization [req-1669e5d3-ce16-40b5-af7e-e6a35bf5611b req-ddcead9d-6465-466b-a890-59092706d20f tempest-AttachVolumeShelveTestJSON-2132759461 tempest-AttachVolumeShelveTestJSON-2132759461-project-member] Enqueued Resource ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 with Priority: HIGHEST for action security_group_members
[  685.825078] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 with Priority: HIGHEST for action security_group_members
[  685.825078] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.825078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.825078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.825078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.825078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.825078] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.825078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.825078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.825637] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.825637] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  685.825637] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  685.825637] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  685.825637] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  685.825892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.825892] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f80ac04b-9522-404c-9dc2-d2e2b8a228d9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  685.825971] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7473415f44174ff884c8d71708ade02b in queue reply_156db6c4256a4c58884afce15e6f5340
[  685.835185] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7473415f44174ff884c8d71708ade02b
[  685.853769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  685.853769] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  685.853769] env[60890]: ERROR oslo_messaging.rpc.server 
[  686.078508] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.078679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.078756] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.078996] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.079115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.079316] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.079439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.079511] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.079671] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.079752] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.079840] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  686.079919] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.207311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  686.207824] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  686.207824] env[60890]: ERROR oslo_messaging.rpc.server 
[  686.273803] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.273997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.274123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.274415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.274562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.274872] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.275044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.275145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.275414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.275678] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.275887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.276021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.276236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.276379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.276531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.276783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.276935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.277055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.277292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.277428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.277670] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.277833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.277948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.278228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.278329] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  686.278574] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8710702611fe4b5f8193c190a3e53f1a in queue reply_156db6c4256a4c58884afce15e6f5340
[  686.285349] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8710702611fe4b5f8193c190a3e53f1a
[  686.328779] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.329090] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.329090] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.329200] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.329290] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.329476] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.329569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.329627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.329773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.329844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.329928] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f80ac04b-9522-404c-9dc2-d2e2b8a228d9 already deleted.
[  686.329999] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f80ac04b-9522-404c-9dc2-d2e2b8a228d9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.535123] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.535123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.535238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.535597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.535869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.536426] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.536711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.536947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.537314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.537735] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.538022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.538253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.538603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.538879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.539310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.539732] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.540106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.540389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.540753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.540950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.541366] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.541366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.541520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.541690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.541830] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  686.542070] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a2865a71ef394362ad4468d8fe7f5cda in queue reply_156db6c4256a4c58884afce15e6f5340
[  686.547894] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2865a71ef394362ad4468d8fe7f5cda
[  686.634165] env[60890]: INFO networking_nsxv3.common.synchronization [req-094f793c-bc74-41fb-8a01-9bef5381cac8 req-43cf73ca-ffce-4abf-99a7-7396e482db24 tempest-SecurityGroupsTestJSON-806588483 tempest-SecurityGroupsTestJSON-806588483-project-member] Enqueued Resource ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 with Priority: HIGHEST for action security_group_members
[  686.635013] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 with Priority: HIGHEST for action security_group_members
[  686.635294] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.635422] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.635484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.635698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.635786] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.635958] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.636075] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.636140] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.636345] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.636490] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.636585] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.636645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.636788] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.636872] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.636957] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('384e1ef2-f71b-4a37-9e25-4fafe57f50e8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  686.637140] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4fbcd0bc7440435eb8697813b1b5c431 in queue reply_156db6c4256a4c58884afce15e6f5340
[  686.652566] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4fbcd0bc7440435eb8697813b1b5c431
[  686.781224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  686.781745] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  686.781745] env[60890]: ERROR oslo_messaging.rpc.server 
[  686.850270] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.850469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.850659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.850934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.851009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.851252] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.851407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.851757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.851757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.851978] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.852398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.852549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.852792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.853098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.853098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.853349] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.853672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.853807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.854024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.854159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  686.854401] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  686.854550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  686.854666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  686.854942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  686.855075] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  686.855304] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg dc805e06efa0464a9dc2fa8bd87ca15e in queue reply_156db6c4256a4c58884afce15e6f5340
[  686.862482] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dc805e06efa0464a9dc2fa8bd87ca15e
[  687.045606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  687.046223] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  687.046223] env[60890]: ERROR oslo_messaging.rpc.server 
[  687.139281] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.139465] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.139523] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.139740] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.139837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.140072] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.140198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.140269] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.140419] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.140501] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.140588] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 already deleted.
[  687.140670] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.248525] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.248668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.248826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.249094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.249235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.249478] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.249624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.249747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.250007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.250242] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.250460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.250503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.250693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.250828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.250970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.251210] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.251352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.251464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.251661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.251785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.252039] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.252191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.252345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.252539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.252687] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  687.252934] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f75cb677f8ff47a3914c0c16b6e91101 in queue reply_156db6c4256a4c58884afce15e6f5340
[  687.269900] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f75cb677f8ff47a3914c0c16b6e91101
[  687.358087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  687.358688] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  687.358688] env[60890]: ERROR oslo_messaging.rpc.server 
[  687.389697] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.389927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.390007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.390284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.390424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.390670] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.390824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.390944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.391159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.391398] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.391552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.391670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.391914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.392106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.392267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.392517] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.392672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.392790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.392997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.393131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.393368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.393516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.393632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.393835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.393973] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  687.394203] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2d3976553acb40d39970e286803cc74c in queue reply_156db6c4256a4c58884afce15e6f5340
[  687.400725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2d3976553acb40d39970e286803cc74c
[  687.762252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  687.763639] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  687.763639] env[60890]: ERROR oslo_messaging.rpc.server 
[  687.777679] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.777882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.778015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.778286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.778433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.778694] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.778853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.778974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.779194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.779437] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.779609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.779700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.779906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.780061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.780236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.780450] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.780600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.780715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.781048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.781048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.781270] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-00d06f9a-5235-439e-a733-c2e400dbd60c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.781420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.781536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.781743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.781881] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('00d06f9a-5235-439e-a733-c2e400dbd60c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  687.782107] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0763109f84a3432e9cf93d6624474f87 in queue reply_156db6c4256a4c58884afce15e6f5340
[  687.794482] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0763109f84a3432e9cf93d6624474f87
[  687.896269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  687.896811] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  687.896811] env[60890]: ERROR oslo_messaging.rpc.server 
[  687.957284] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.957491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.957624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.957929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.958118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.958371] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.958549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.958678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.958913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.959181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.959353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.959488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.959717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.959904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.960091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.960374] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.960549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.960673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.960888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.961053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  687.961328] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  687.961568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  687.961720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  687.961955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  687.962114] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  687.962363] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9c61df026c3542c59c18764487ab67af in queue reply_156db6c4256a4c58884afce15e6f5340
[  687.966926] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9c61df026c3542c59c18764487ab67af
[  688.284233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-00d06f9a-5235-439e-a733-c2e400dbd60c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  688.284888] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  688.284888] env[60890]: ERROR oslo_messaging.rpc.server 
[  688.394575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.394883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.394973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.395261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.395376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.395636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.395866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.395962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.396233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.396482] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.396643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.396764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.396977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.397153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.397315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.398047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.398047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.398047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.398047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.398520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.398520] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.398520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.398831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.398831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.399008] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  688.399409] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c53d5786883b4c01b88db234927b39cf in queue reply_156db6c4256a4c58884afce15e6f5340
[  688.404498] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c53d5786883b4c01b88db234927b39cf
[  688.464138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  688.464588] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  688.464588] env[60890]: ERROR oslo_messaging.rpc.server 
[  688.659060] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.659282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.659460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.659684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.659831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.660149] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.660329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.660451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.660672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.660924] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.661082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.661201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.661475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.661543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.661686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.661941] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.662099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.662223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.662417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.662550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.662842] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  688.663009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  688.663129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  688.663341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  688.663479] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  688.663715] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8e94a374a06e4f7e92ac0ef06680c323 in queue reply_156db6c4256a4c58884afce15e6f5340
[  688.669448] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8e94a374a06e4f7e92ac0ef06680c323
[  688.902908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  688.902908] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  688.902908] env[60890]: ERROR oslo_messaging.rpc.server 
[  689.062196] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.062410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.062538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.062843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.063000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.063262] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.063419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.063538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.063755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.064400] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.064400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.064400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.064590] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.064694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.064825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.065079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.065234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.065351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.065559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.065694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.065931] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.066081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.066196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.066399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.066534] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  689.066759] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3a766a8edca147648a94f69a60397c92 in queue reply_156db6c4256a4c58884afce15e6f5340
[  689.071660] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3a766a8edca147648a94f69a60397c92
[  689.166257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  689.166753] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  689.166753] env[60890]: ERROR oslo_messaging.rpc.server 
[  689.216383] env[60890]: INFO networking_nsxv3.common.synchronization [req-904c238b-5702-4769-898d-0e206101cbfb req-05aeae37-a13b-44d9-81bc-3ef0e443d93c tempest-ServerAddressesTestJSON-521927788 tempest-ServerAddressesTestJSON-521927788-project-member] Enqueued Resource ID: 458f359e-9b52-473c-8206-6b4996c8c6d7 with Priority: HIGHEST for action security_group_members
[  689.217484] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 458f359e-9b52-473c-8206-6b4996c8c6d7 with Priority: HIGHEST for action security_group_members
[  689.217484] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.217484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.217484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.217998] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.217998] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.217998] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-458f359e-9b52-473c-8206-6b4996c8c6d7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.217998] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-458f359e-9b52-473c-8206-6b4996c8c6d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.218151] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-458f359e-9b52-473c-8206-6b4996c8c6d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.218228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-458f359e-9b52-473c-8206-6b4996c8c6d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.218376] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.218454] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.218526] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.218871] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.218968] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.222140] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('458f359e-9b52-473c-8206-6b4996c8c6d7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  689.222140] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f85acb0678604bbca2a6212c86c66d6d in queue reply_156db6c4256a4c58884afce15e6f5340
[  689.231508] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f85acb0678604bbca2a6212c86c66d6d
[  689.568983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  689.569577] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  689.569577] env[60890]: ERROR oslo_messaging.rpc.server 
[  689.721774] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.722037] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.722155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.722404] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.722515] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.722739] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.722837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.722903] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.723070] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.723153] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.723243] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 458f359e-9b52-473c-8206-6b4996c8c6d7 already deleted.
[  689.723323] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-458f359e-9b52-473c-8206-6b4996c8c6d7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.798882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.798882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.798882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.798882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.798882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.798882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.798882] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  689.798882] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg aea08b477dac481486a1e6285a7f02ac in queue reply_156db6c4256a4c58884afce15e6f5340
[  689.806796] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg aea08b477dac481486a1e6285a7f02ac
[  689.857559] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.857854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.858035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.858390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.858617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.858999] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.859256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.859757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.859757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.860157] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.860337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.860509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.860846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.861074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.861325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.861770] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.861862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.862009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.862267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.862469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  689.862835] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  689.863038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  689.863407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  689.863537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  689.863746] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  689.864125] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5c6f658ab6954e54b7b8f89ddb70173d in queue reply_156db6c4256a4c58884afce15e6f5340
[  689.869905] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5c6f658ab6954e54b7b8f89ddb70173d
[  690.144421] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.144639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.144753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.145031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.145216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.145455] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-699 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.145613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.145734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.145962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.146208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.146365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.146483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.146693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.146843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.146992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-699" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.147287] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.147449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.147568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.147780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.147921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.148194] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fcad0492-c8d4-4285-8ebd-11425593857b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.148354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.148471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.148681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.148818] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fcad0492-c8d4-4285-8ebd-11425593857b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  690.149055] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4e771ce015d64e6cb601865d29b20929 in queue reply_156db6c4256a4c58884afce15e6f5340
[  690.154632] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4e771ce015d64e6cb601865d29b20929
[  690.303358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  690.303870] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  690.303870] env[60890]: ERROR oslo_messaging.rpc.server 
[  690.366885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  690.367451] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  690.367451] env[60890]: ERROR oslo_messaging.rpc.server 
[  690.651887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fcad0492-c8d4-4285-8ebd-11425593857b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  690.652410] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  690.652410] env[60890]: ERROR oslo_messaging.rpc.server 
[  690.882342] env[60890]: INFO networking_nsxv3.common.synchronization [req-8d40c603-389f-4b43-a417-3974f8dcf6f0 req-71e92527-098e-40f2-8072-bd619f40525e tempest-ImagesTestJSON-598093153 tempest-ImagesTestJSON-598093153-project-member] Enqueued Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  690.882909] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  690.883230] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.883357] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.883425] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.883619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.883701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.883847] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f0a2229c-87aa-4f25-9157-b1b2e986b23e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.883943] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.884016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.884181] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.884325] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.884418] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.884478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.884767] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.884767] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.884893] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f0a2229c-87aa-4f25-9157-b1b2e986b23e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  690.884964] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7c1ed398bbc64e1785808bf12ede76ef in queue reply_156db6c4256a4c58884afce15e6f5340
[  690.892084] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7c1ed398bbc64e1785808bf12ede76ef
[  690.947324] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.947535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.947664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.947936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.948110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.948349] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.948497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.948610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.948824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.949064] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.949216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.949332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.949538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.949683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.949830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.950078] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.950233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.950350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.950551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.950683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.950918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.951071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.951185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.951389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.951519] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  690.951750] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e129b2d689f749b99536c44e39223be5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  690.958332] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e129b2d689f749b99536c44e39223be5
[  690.987054] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.987255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.987384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.987642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.987788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.988056] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.988208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.988325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.988838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.990115] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.990115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.990115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.990115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.990115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.990115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.990413] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.990466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.990569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.990859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.991014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  690.991254] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  690.991404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  690.991631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  690.991857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  690.991996] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  690.992250] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9379414db29e4e7fa15caba804bc31b7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  690.998601] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9379414db29e4e7fa15caba804bc31b7
[  691.200692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.200692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.200692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.200692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.201283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.201487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.201698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.202007] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.202231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.202731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.203023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.203231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.203532] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.203755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.203933] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.204253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.204702] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  691.205041] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c49cf11cde834395b62202235bf908e8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  691.210432] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c49cf11cde834395b62202235bf908e8
[  691.386552] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.386708] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.386773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.387001] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.387089] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.387282] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.387375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.387433] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.387581] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.387652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.387742] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e already deleted.
[  691.387818] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.454109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  691.454838] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  691.454838] env[60890]: ERROR oslo_messaging.rpc.server 
[  691.462181] env[60890]: INFO networking_nsxv3.common.synchronization [req-e9cf7b58-3f2b-435a-8967-4678520957f4 req-8c4231d0-bcc0-4901-88ca-c10efd5f71ee tempest-ServerMetadataTestJSON-491230402 tempest-ServerMetadataTestJSON-491230402-project-member] Enqueued Resource ID: 617466de-da46-4007-bf41-f1add494a4d0 with Priority: HIGHEST for action security_group_members
[  691.462855] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 617466de-da46-4007-bf41-f1add494a4d0 with Priority: HIGHEST for action security_group_members
[  691.463222] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.463363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.463434] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.463637] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.463726] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.463883] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-617466de-da46-4007-bf41-f1add494a4d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.463982] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-617466de-da46-4007-bf41-f1add494a4d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.464402] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-617466de-da46-4007-bf41-f1add494a4d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.464621] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-617466de-da46-4007-bf41-f1add494a4d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.464877] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.465034] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.465142] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.465336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.465473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.465611] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('617466de-da46-4007-bf41-f1add494a4d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  691.465847] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3b213d5f9d194efcb6c0f2f9ba9faf32 in queue reply_156db6c4256a4c58884afce15e6f5340
[  691.474630] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3b213d5f9d194efcb6c0f2f9ba9faf32
[  691.495529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  691.496034] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  691.496034] env[60890]: ERROR oslo_messaging.rpc.server 
[  691.708253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  691.708797] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  691.708797] env[60890]: ERROR oslo_messaging.rpc.server 
[  691.969362] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.969512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.969586] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.969811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.969908] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.970264] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  691.970264] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  691.970264] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  691.970444] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  691.970481] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  691.970561] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 617466de-da46-4007-bf41-f1add494a4d0 already deleted.
[  691.970640] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-617466de-da46-4007-bf41-f1add494a4d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.093063] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.093270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.093402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.093665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.093814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.094073] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.094227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.094349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.094587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.094876] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.095037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.095154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.095362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.095500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.095646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.097055] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.097055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.097055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.097055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.097055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.097055] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.097055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.097055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.097465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.097465] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  692.097767] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ded79c72632f47858021d60745652c52 in queue reply_156db6c4256a4c58884afce15e6f5340
[  692.103850] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ded79c72632f47858021d60745652c52
[  692.114089] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.114295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.114424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.114672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.115125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.115125] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.115302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.115405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.115656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.116056] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.116229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.116351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.116561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.116708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.116858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.117106] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.117261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.117379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.117583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.117717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.117951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.118103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.118222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.118426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.118559] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  692.118787] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ec2003c0253842df927c2adb64489bb4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  692.124985] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ec2003c0253842df927c2adb64489bb4
[  692.391028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.391263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.391396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.391667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.391817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.392096] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.392256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.392377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.392634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.393088] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.393274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.393404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.393629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.393777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.393934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.394197] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.394354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.394476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.394776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.394942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.395188] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.395344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.395464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.395671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.395873] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  692.396215] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cf1063f46b444e09867a5fb025a264d9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  692.401428] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cf1063f46b444e09867a5fb025a264d9
[  692.599609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  692.600147] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  692.600147] env[60890]: ERROR oslo_messaging.rpc.server 
[  692.622399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  692.622399] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  692.622399] env[60890]: ERROR oslo_messaging.rpc.server 
[  692.767566] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.767872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.767916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.768231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.768383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.768651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.768805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.768924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.769145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.769390] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.769546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.769667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.769874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.770020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.770166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.770414] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.770567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.770683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.770889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.771026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.771265] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  692.771414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  692.771533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  692.771739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  692.771877] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  692.772251] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9cbf908e7fec4cb6a3a6fea92fddffb8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  692.778533] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9cbf908e7fec4cb6a3a6fea92fddffb8
[  692.897898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  692.898396] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  692.898396] env[60890]: ERROR oslo_messaging.rpc.server 
[  693.176803] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.177016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.177169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.177445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.177595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.177853] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.178040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.178170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.178394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.178640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.178795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.178911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.179683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.179840] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.179996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.180271] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.180428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.180539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.180751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.180896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.181181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.181343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.181462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.181671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.181806] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  693.182039] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 170b3cb985ff4e94be625c39d6a357fb in queue reply_156db6c4256a4c58884afce15e6f5340
[  693.190072] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 170b3cb985ff4e94be625c39d6a357fb
[  693.274496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  693.275044] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  693.275044] env[60890]: ERROR oslo_messaging.rpc.server 
[  693.359359] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.359571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.359703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.359974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.360147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.360404] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.360592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.360685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.360905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.361153] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.361309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.361429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.361639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.361852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.362011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.362262] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.362417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.362537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.362746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.362884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.363204] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.363388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.363514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.363726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.363867] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  693.364116] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ad4d6aaf3409474bb482613f6232948a in queue reply_156db6c4256a4c58884afce15e6f5340
[  693.422249] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ad4d6aaf3409474bb482613f6232948a
[  693.487891] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.488163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.488300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.488571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.488724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.489034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.489146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.489285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.489451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.489704] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.489860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.489980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.490189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.490335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.490483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.490738] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.490892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.491013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.491222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.491354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.491593] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  693.491748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  693.491863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  693.492086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  693.492224] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  693.492452] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f071ba9539ee470c918d77d2ad24cae8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  693.520304] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f071ba9539ee470c918d77d2ad24cae8
[  693.684299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  693.684892] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  693.684892] env[60890]: ERROR oslo_messaging.rpc.server 
[  693.866744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  693.867319] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  693.867319] env[60890]: ERROR oslo_messaging.rpc.server 
[  693.994437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  693.995007] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  693.995007] env[60890]: ERROR oslo_messaging.rpc.server 
[  694.422463] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.422463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.422463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.422463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.422463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.422463] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.422463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.422463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.422463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.423651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.423651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.423651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.423651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.423651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.424029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.424029] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.424280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.424335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.424504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.424644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.424963] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.425125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.425249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.425472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.425642] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  694.426009] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a896d09ef2e5460aaaa039e47fad8072 in queue reply_156db6c4256a4c58884afce15e6f5340
[  694.441139] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a896d09ef2e5460aaaa039e47fad8072
[  694.495990] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.495990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.496295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.496428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.496578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.496954] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.497114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.497232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.497475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.497690] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.497837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.497952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.498258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.498398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.498546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.498793] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.498939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.499057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.499258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.499384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.499616] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.499762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.499876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.500090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.500222] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  694.500443] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f3c553ef9c4a45368b5e1de34d0df312 in queue reply_156db6c4256a4c58884afce15e6f5340
[  694.505523] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f3c553ef9c4a45368b5e1de34d0df312
[  694.832036] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.832262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.832443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.833981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.833981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.833981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.833981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.833981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.833981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.833981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.833981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.834340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.834340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.834406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.836032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.836032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.836032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.836032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.836032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.836032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.836032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  694.836032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  694.836032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  694.836032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  694.836474] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  694.836474] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2742d3b21748430baeaa602893293d42 in queue reply_156db6c4256a4c58884afce15e6f5340
[  694.841182] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2742d3b21748430baeaa602893293d42
[  694.928614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  694.929132] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  694.929132] env[60890]: ERROR oslo_messaging.rpc.server 
[  695.003712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  695.004226] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  695.004226] env[60890]: ERROR oslo_messaging.rpc.server 
[  695.356827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  695.356827] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  695.356827] env[60890]: ERROR oslo_messaging.rpc.server 
[  695.535420] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.535653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.535764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.536061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.536215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.536477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.536662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.536758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.537018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.537240] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.537378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.537496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.537701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.537850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.538015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.538238] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.538387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.538503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.538708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.538851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.539149] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.539267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.539395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.539592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.539733] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  695.540012] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3de73755d41143ad9ffe890d44f0b924 in queue reply_156db6c4256a4c58884afce15e6f5340
[  695.546990] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3de73755d41143ad9ffe890d44f0b924
[  695.591691] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.591890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.592034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.592400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.592565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.592866] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.593025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.593143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.593368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.593610] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.593756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.593913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.594166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.594863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.595056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.595317] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.595475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.595593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.595886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.596045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.596293] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.596443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.596560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.596765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.596920] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('796f7ca0-828a-4d97-b1c9-6cd12fe12b95',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  695.597165] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e55a8a1ff1044f9ea157663c26c2c839 in queue reply_156db6c4256a4c58884afce15e6f5340
[  695.603321] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e55a8a1ff1044f9ea157663c26c2c839
[  695.945698] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.946014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.946047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.946312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.946461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.946728] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.946896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.946991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.947230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.947768] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.947950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.948083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.948299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.948449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.948601] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.948852] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.949007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.949128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.949335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.949470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  695.949710] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  695.949869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  695.950007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  695.950272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  695.950421] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  695.950653] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 539133c8c2014eb4bf91e12786912159 in queue reply_156db6c4256a4c58884afce15e6f5340
[  695.956075] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 539133c8c2014eb4bf91e12786912159
[  696.043950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  696.043950] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  696.043950] env[60890]: ERROR oslo_messaging.rpc.server 
[  696.099779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-796f7ca0-828a-4d97-b1c9-6cd12fe12b95" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  696.099779] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  696.099779] env[60890]: ERROR oslo_messaging.rpc.server 
[  696.453087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  696.453595] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  696.453595] env[60890]: ERROR oslo_messaging.rpc.server 
[  696.463403] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  696.463495] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  696.944268] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  696.944268] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  696.944268] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  696.944268] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  696.944268] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  696.944268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  696.944268] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  696.944268] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7e1f09b16a194264ae5885423d45e31b in queue reply_156db6c4256a4c58884afce15e6f5340
[  696.948521] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7e1f09b16a194264ae5885423d45e31b
[  697.038740] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.038949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.039135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.039344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.039495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.039755] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.039923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.040047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.040274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.040523] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.040681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.040802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.041017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.041162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.041311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.041562] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.041829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.041829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.042036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.042168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.042400] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.042553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.042666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.042912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.043052] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  697.043279] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f582712f830144fe85ee22a996797c73 in queue reply_156db6c4256a4c58884afce15e6f5340
[  697.048765] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f582712f830144fe85ee22a996797c73
[  697.120373] env[60890]: INFO networking_nsxv3.common.synchronization [req-3c66067b-6b9c-4c02-a034-e5f2c1a1ae1c req-aca79c8e-ef81-4a0c-b94a-7f6a494251c8 tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  697.120373] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  697.120373] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.120373] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.120373] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.120373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.120373] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  697.120373] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 22c27fd819bc4296bb96f8ff6b11b302 in queue reply_156db6c4256a4c58884afce15e6f5340
[  697.128231] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 22c27fd819bc4296bb96f8ff6b11b302
[  697.202677] env[60890]: INFO networking_nsxv3.common.synchronization [req-aac9b989-cb2a-487f-9f8f-6df3dcc8da66 req-09bc8e90-1054-4c65-bdd2-64ae4683e4d2 tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 with Priority: HIGHEST for action security_group_members
[  697.202943] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 with Priority: HIGHEST for action security_group_members
[  697.203481] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.203606] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.203674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.203871] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.203957] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.204133] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.204226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.204331] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.204466] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.204605] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.204697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.204777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.204921] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.205000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.205089] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  697.205270] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1fec9a198b5049238f47213bc2ddf2b5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  697.217289] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1fec9a198b5049238f47213bc2ddf2b5
[  697.444514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  697.445055] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  697.445055] env[60890]: ERROR oslo_messaging.rpc.server 
[  697.531974] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.532227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.533317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.533660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.533821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.534476] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.534655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.534800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.535080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.535335] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.535490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.535609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.535914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.536087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.536263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.536494] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.536645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.536762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.536969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.537103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.537341] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.537519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.537640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.537890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.538025] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  697.538260] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 730b30c4e7e6427581fc095245b16129 in queue reply_156db6c4256a4c58884afce15e6f5340
[  697.546826] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 730b30c4e7e6427581fc095245b16129
[  697.546826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  697.546826] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  697.546826] env[60890]: ERROR oslo_messaging.rpc.server 
[  697.623550] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.623550] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.623550] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  697.623550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.711027] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.711027] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  697.711027] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 already deleted.
[  697.711027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.041147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  698.041674] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  698.041674] env[60890]: ERROR oslo_messaging.rpc.server 
[  698.135242] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.135481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.135609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.135909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.136282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.136347] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.136484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.136599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.136842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.137086] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.137237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.137352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.137563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.137707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.137889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.138165] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.138335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.138463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.138677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.138815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.139050] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.139223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.139343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.139548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.139686] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  698.139975] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4e15a4823ef44e73a6e3844d23613d07 in queue reply_156db6c4256a4c58884afce15e6f5340
[  698.144907] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4e15a4823ef44e73a6e3844d23613d07
[  698.575114] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.575323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.575447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.575748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.575902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.576239] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.576422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.576544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.576785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.577058] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.577199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.577318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.577527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.577671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.577821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.578073] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.578225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.578342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.578548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.578680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.578918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-71fa3df2-9519-4d0e-bb8e-06d177537bbc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.579129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.579277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.579495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.579630] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('71fa3df2-9519-4d0e-bb8e-06d177537bbc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  698.579872] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg aeb8ac924a39415bb34d0a870343c237 in queue reply_156db6c4256a4c58884afce15e6f5340
[  698.585256] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg aeb8ac924a39415bb34d0a870343c237
[  698.642925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  698.643497] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  698.643497] env[60890]: ERROR oslo_messaging.rpc.server 
[  698.887322] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.887518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.887640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.887916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.888084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.888336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.888489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.888604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.888853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.889107] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.889256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.889370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.889576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.889717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.889863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.890152] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.890303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.890421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.890623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.890758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.890994] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.891142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.891257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.891457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.891589] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  698.891982] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 896513db82064050aad984ba5c0f39a0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  698.893416] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.893582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.893701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.893912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.894046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.894281] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.894429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.894544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.894805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.895055] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.895206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.895320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.895521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.895673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.895870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.896148] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.896300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.896413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.896616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.896743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.896973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  698.900480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  698.900565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  698.900783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  698.900916] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  698.901138] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6cb1b56803fe41e4a58332913c5a8950 in queue reply_156db6c4256a4c58884afce15e6f5340
[  698.902461] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 896513db82064050aad984ba5c0f39a0
[  698.903155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  698.903570] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  698.903570] env[60890]: ERROR oslo_messaging.rpc.server 
[  698.906993] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6cb1b56803fe41e4a58332913c5a8950
[  699.083757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-71fa3df2-9519-4d0e-bb8e-06d177537bbc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  699.083757] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  699.083757] env[60890]: ERROR oslo_messaging.rpc.server 
[  699.405603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  699.405603] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  699.405603] env[60890]: ERROR oslo_messaging.rpc.server 
[  699.560218] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.560218] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.560218] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.560218] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.560218] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.560218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.560218] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  699.560218] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8118a281115c4b88a3ce14b20e1635b7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  699.565710] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8118a281115c4b88a3ce14b20e1635b7
[  699.685294] env[60890]: INFO networking_nsxv3.common.synchronization [req-3c66067b-6b9c-4c02-a034-e5f2c1a1ae1c req-16f38cda-5cb6-48e8-98f8-b59e73a12214 tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  699.687186] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  699.687186] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.687472] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.687472] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.688464] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.688464] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.688464] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.688464] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.688464] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.688464] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.688464] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.688711] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.690262] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.690262] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.690262] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.690262] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  699.690262] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 959c591dc2d14341bab5df252a3cf66e in queue reply_156db6c4256a4c58884afce15e6f5340
[  699.698940] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 959c591dc2d14341bab5df252a3cf66e
[  699.880282] env[60890]: INFO networking_nsxv3.common.synchronization [req-3c66067b-6b9c-4c02-a034-e5f2c1a1ae1c req-ce68727f-1b86-42cb-99a9-f21691c7c5c9 tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  699.880808] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  699.881139] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.881265] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  699.881336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  699.881536] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  699.881622] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  699.881774] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  699.881900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.016971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.017224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.017394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.017693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.017977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.018282] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.018482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.018636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.018897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.019193] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.019392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.019548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.019798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.019987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.020187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.020476] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.020676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.020833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.021087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.021249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.021517] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.021714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.021871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.022114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.022291] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  700.022561] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 53598714a68f4141bb16796042260c78 in queue reply_156db6c4256a4c58884afce15e6f5340
[  700.028991] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 53598714a68f4141bb16796042260c78
[  700.062905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  700.063425] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  700.063425] env[60890]: ERROR oslo_messaging.rpc.server 
[  700.191012] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.191250] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.191250] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.191454] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.191549] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.191740] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.191842] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.191906] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.192075] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.192155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.192242] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  700.192321] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.192480] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.192626] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.192767] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.192861] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.192973] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.193053] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.193172] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.193214] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  700.193399] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9795dfdbf5ae4b35be6806ff75c6b537 in queue reply_156db6c4256a4c58884afce15e6f5340
[  700.203713] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9795dfdbf5ae4b35be6806ff75c6b537
[  700.256304] env[60890]: INFO networking_nsxv3.common.synchronization [req-094f793c-bc74-41fb-8a01-9bef5381cac8 req-c94ca4f5-b716-41a9-993d-476a03e53a69 tempest-SecurityGroupsTestJSON-806588483 tempest-SecurityGroupsTestJSON-806588483-project-member] Enqueued Resource ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 with Priority: HIGHEST for action security_group_members
[  700.256790] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 with Priority: HIGHEST for action security_group_members
[  700.257111] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.257238] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.257297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.257491] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.257578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.257726] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.257822] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.257883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.258029] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.258168] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.258259] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.258319] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.258454] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.258533] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.258621] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('384e1ef2-f71b-4a37-9e25-4fafe57f50e8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  700.258975] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a2ada66151c744019acd7d5f54ae4828 in queue reply_156db6c4256a4c58884afce15e6f5340
[  700.274153] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2ada66151c744019acd7d5f54ae4828
[  700.525426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  700.525676] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  700.525676] env[60890]: ERROR oslo_messaging.rpc.server 
[  700.664803] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.665037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.665170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.665440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.665591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.665857] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.666015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.666184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.666348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.666593] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.666749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.666870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.667103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.667260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.667413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.667663] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.667819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.667939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.668192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.668332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.668572] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-efc6d8c3-031b-49ef-8f18-ff41b0db232e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.668727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.668848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.669057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.669197] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('efc6d8c3-031b-49ef-8f18-ff41b0db232e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  700.669427] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d4dd172a100f426f8bc6b1451236da74 in queue reply_156db6c4256a4c58884afce15e6f5340
[  700.675160] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d4dd172a100f426f8bc6b1451236da74
[  700.695850] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.696121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.696194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.696414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.696513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.696713] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.696821] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.696885] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.697044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.697126] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.697212] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  700.697294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.760772] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.760930] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.761003] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.761228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.761332] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.761528] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  700.761631] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  700.761696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  700.761850] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  700.761931] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  700.762019] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 already deleted.
[  700.762099] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  701.171378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-efc6d8c3-031b-49ef-8f18-ff41b0db232e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  701.171894] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  701.171894] env[60890]: ERROR oslo_messaging.rpc.server 
[  701.325458] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  701.325760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  701.325793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  701.326060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  701.326211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  701.326476] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  701.326628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  701.326754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  701.326978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  701.327243] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  701.327415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  701.327539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  701.327752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  701.327898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  701.328066] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  701.328337] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  701.328509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  701.328631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  701.328877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  701.329023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  701.329333] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  701.329495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  701.329617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  701.329830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  701.329971] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  701.330206] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fff681de2ca043a9b16f09d5b56608ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  701.335738] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fff681de2ca043a9b16f09d5b56608ad
[  701.832409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  701.832932] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  701.832932] env[60890]: ERROR oslo_messaging.rpc.server 
[  702.191787] env[60890]: INFO networking_nsxv3.common.synchronization [req-d2f0426c-c76d-4fd2-8549-ae020bed1361 req-bdc41f53-b06c-49a3-a8f5-8290c30750cf tempest-AttachVolumeTestJSON-389769224 tempest-AttachVolumeTestJSON-389769224-project-member] Enqueued Resource ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 with Priority: HIGHEST for action security_group_members
[  702.193149] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 with Priority: HIGHEST for action security_group_members
[  702.193149] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.193149] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.193149] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.193149] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.193149] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.194789] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-73956ef6-bfe0-49e9-80aa-feabeac5ab67 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.194789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.194789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.194789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.194789] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.194789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.194789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.194789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.194789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.194789] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('73956ef6-bfe0-49e9-80aa-feabeac5ab67',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  702.194789] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 24325d1f8e554f12b8fb4f8f257f02e1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  702.215805] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 24325d1f8e554f12b8fb4f8f257f02e1
[  702.698370] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.698666] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.698666] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.699038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.699038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.699151] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.699251] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.699314] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.699472] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.699551] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.699636] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 already deleted.
[  702.699716] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.848763] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.848973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.849102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.849375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.849523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.849780] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.849974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.850055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.850266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.850507] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.850659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.850779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.850988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.851128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.851276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.851524] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.851676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.851794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.852016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.853009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.853274] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.853512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.853549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.853772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.853913] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  702.854144] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1b09899e0aaf4f6e837fb3d57bcdcf91 in queue reply_156db6c4256a4c58884afce15e6f5340
[  702.860235] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1b09899e0aaf4f6e837fb3d57bcdcf91
[  702.962932] env[60890]: INFO networking_nsxv3.common.synchronization [None req-064a96b9-b975-4655-87a0-b1cd4dadb47b None None] Enqueued Resource ID: 10530114-7f07-4e0d-8db4-9970dc874e7d with Priority: HIGHEST for action port
[  702.963193] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 10530114-7f07-4e0d-8db4-9970dc874e7d with Priority: HIGHEST for action port
[  702.964091] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.964304] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.964304] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.964781] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.964781] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.964781] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-10530114-7f07-4e0d-8db4-9970dc874e7d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.964916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-10530114-7f07-4e0d-8db4-9970dc874e7d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.964916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-10530114-7f07-4e0d-8db4-9970dc874e7d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.965089] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-10530114-7f07-4e0d-8db4-9970dc874e7d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.965170] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('10530114-7f07-4e0d-8db4-9970dc874e7d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  702.965348] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 44f66879e3a0416a83f09c1525b3f66b in queue reply_156db6c4256a4c58884afce15e6f5340
[  702.970394] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 44f66879e3a0416a83f09c1525b3f66b
[  702.993489] env[60890]: INFO networking_nsxv3.common.synchronization [req-8d40c603-389f-4b43-a417-3974f8dcf6f0 req-322c8937-01cd-49e6-8c6a-2c1c05069d16 tempest-ImagesTestJSON-598093153 tempest-ImagesTestJSON-598093153-project-member] Enqueued Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  702.994037] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  702.994350] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.994479] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.994542] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.994774] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.994854] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.994987] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f0a2229c-87aa-4f25-9157-b1b2e986b23e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.995074] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.995178] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.995277] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.995409] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  702.995556] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  702.995618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  702.995779] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  702.995920] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  702.996030] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f0a2229c-87aa-4f25-9157-b1b2e986b23e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  702.996623] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 25d19b9b30dc44f1899b567e5d65b9c0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  703.020570] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 25d19b9b30dc44f1899b567e5d65b9c0
[  703.358242] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  703.358775] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  703.358775] env[60890]: ERROR oslo_messaging.rpc.server 
[  703.437855] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  703.438057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  703.438182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  703.438443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  703.438583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.438845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  703.438996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  703.439110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  703.439375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  703.439613] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  703.439762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  703.439877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  703.440097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  703.440236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.440378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.440618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  703.440766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  703.440879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  703.441076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  703.441203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.441433] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  703.441576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  703.441688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  703.441944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  703.442072] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  703.442300] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 780db6d0fd6a43de9cdc51ba38018cfc in queue reply_156db6c4256a4c58884afce15e6f5340
[  703.457934] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 780db6d0fd6a43de9cdc51ba38018cfc
[  703.467573] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-10530114-7f07-4e0d-8db4-9970dc874e7d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.467801] env[60890]: Traceback (most recent call last):
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  703.467863] env[60890]:     timer()
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  703.467863] env[60890]:     cb(*args, **kw)
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  703.467863] env[60890]:     result = function(*args, **kwargs)
[  703.467863] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  703.467863] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  703.467863] env[60890]:     return method(*args, **kwargs)
[  703.467863] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  703.467863] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  703.467863] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  703.467863] env[60890]:     result = self.transport._send(
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  703.467863] env[60890]:     return self._driver.send(target, ctxt, message,
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  703.467863] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  703.467863] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  703.467863] env[60890]:     raise result
[  703.467863] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  703.467863] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  703.499125] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  703.499276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  703.499344] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  703.499556] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  703.499646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.499848] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  703.499943] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  703.500018] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  703.500361] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  703.500361] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.500361] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e already deleted.
[  703.500449] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.945887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  703.945887] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  703.945887] env[60890]: ERROR oslo_messaging.rpc.server 
[  704.380799] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.381034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.381162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.381437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.381580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.381845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.381997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.382115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.382335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.382582] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.382787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.382918] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.383130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.383270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.383417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.383665] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.383825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.383940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.384200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.384473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.384569] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.384724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.384868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.385084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.385310] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  704.385453] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 47760d7aa7f2456cbe95ad0dafa33e39 in queue reply_156db6c4256a4c58884afce15e6f5340
[  704.391350] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 47760d7aa7f2456cbe95ad0dafa33e39
[  704.447404] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.447615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.447745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.448047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.448195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.448470] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.448622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.448742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.448965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.449214] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.449374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.449495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.449721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.449872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.450021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.450272] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.450425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.450796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.451047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.451189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.451433] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.451587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.452258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.452506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.452654] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  704.453010] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 56005b0e121641cfab7f9b740761b3cf in queue reply_156db6c4256a4c58884afce15e6f5340
[  704.459045] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 56005b0e121641cfab7f9b740761b3cf
[  704.486451] env[60890]: INFO networking_nsxv3.common.synchronization [None req-0dae823b-6efb-4c39-a050-b5e6d75bd998 None None] Enqueued Resource ID: 280043bf-d7f1-4fd5-8163-bf10417e5070 with Priority: HIGHEST for action port
[  704.486718] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 280043bf-d7f1-4fd5-8163-bf10417e5070 with Priority: HIGHEST for action port
[  704.487285] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.487432] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.487466] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.487669] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.487758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.487916] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-280043bf-d7f1-4fd5-8163-bf10417e5070 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  704.488060] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-280043bf-d7f1-4fd5-8163-bf10417e5070" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  704.488107] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-280043bf-d7f1-4fd5-8163-bf10417e5070" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  704.488297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-280043bf-d7f1-4fd5-8163-bf10417e5070" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  704.488377] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('280043bf-d7f1-4fd5-8163-bf10417e5070',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  704.488560] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4f8d2767f19f483c91b7f271e9b833ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  704.494541] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4f8d2767f19f483c91b7f271e9b833ad
[  704.887531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  704.888084] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  704.888084] env[60890]: ERROR oslo_messaging.rpc.server 
[  704.955288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  704.955848] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  704.955848] env[60890]: ERROR oslo_messaging.rpc.server 
[  704.990226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-280043bf-d7f1-4fd5-8163-bf10417e5070" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  704.990387] env[60890]: Traceback (most recent call last):
[  704.990423] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  704.990423] env[60890]:     timer()
[  704.990423] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  704.990423] env[60890]:     cb(*args, **kw)
[  704.990423] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  704.990423] env[60890]:     result = function(*args, **kwargs)
[  704.990423] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  704.990423] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  704.990423] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  704.990423] env[60890]:     return method(*args, **kwargs)
[  704.990682] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  704.990682] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  704.990682] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  704.990682] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  704.990682] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  704.990682] env[60890]:     result = self.transport._send(
[  704.990682] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  704.990682] env[60890]:     return self._driver.send(target, ctxt, message,
[  704.990682] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  704.990682] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  704.990682] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  704.990682] env[60890]:     raise result
[  704.990682] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  704.990682] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  705.314384] env[60890]: INFO networking_nsxv3.common.synchronization [None req-e3ddca95-5e7c-4d18-98ff-ec90bbb24d73 None None] Enqueued Resource ID: 2691167b-8e2d-4d39-9fe1-69715b7bc23b with Priority: HIGHEST for action port
[  705.315039] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2691167b-8e2d-4d39-9fe1-69715b7bc23b with Priority: HIGHEST for action port
[  705.315359] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.315576] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.315576] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.315729] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.315819] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.315981] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2691167b-8e2d-4d39-9fe1-69715b7bc23b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.316095] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2691167b-8e2d-4d39-9fe1-69715b7bc23b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.316154] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2691167b-8e2d-4d39-9fe1-69715b7bc23b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.316373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2691167b-8e2d-4d39-9fe1-69715b7bc23b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.316431] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2691167b-8e2d-4d39-9fe1-69715b7bc23b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  705.316610] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 27d4f10857b3493d81345e96782ef89f in queue reply_156db6c4256a4c58884afce15e6f5340
[  705.329367] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 27d4f10857b3493d81345e96782ef89f
[  705.460034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.460253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.460377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.460680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.460783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.461151] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.461235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.461309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.461527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.461797] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.461924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.462045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.462296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.462394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.462542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.463600] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.463880] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.464032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.464256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.464391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.464633] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.465011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.465011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.465134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.465351] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  705.465574] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 30dd77920d514e3f96e81b3d6070bb44 in queue reply_156db6c4256a4c58884afce15e6f5340
[  705.480911] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 30dd77920d514e3f96e81b3d6070bb44
[  705.550832] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.551010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.551134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.551394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.551547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.551805] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.551952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.552092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.552307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.552555] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.552720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.552855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.553058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.553198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.553338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.553578] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.553736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.553850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.554049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.554176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.554404] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.554550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.554663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.554895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.555027] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  705.555254] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5d82108fe5df432aa17bffef76869435 in queue reply_156db6c4256a4c58884afce15e6f5340
[  705.570795] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5d82108fe5df432aa17bffef76869435
[  705.807381] env[60890]: INFO networking_nsxv3.common.synchronization [None req-50ea5e09-cd43-4a5e-a18d-72bc38c5b2a8 tempest-ServerActionsV293TestJSON-1045119592 tempest-ServerActionsV293TestJSON-1045119592-project-member] Enqueued Resource ID: e717cb10-dc75-40d9-a823-032d7367566c with Priority: HIGHEST for action security_group_rules
[  705.807885] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e717cb10-dc75-40d9-a823-032d7367566c with Priority: HIGHEST for action security_group_rules
[  705.808231] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.808355] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.808426] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.808618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.808698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.808851] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-e717cb10-dc75-40d9-a823-032d7367566c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.808967] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.809009] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.809206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.809280] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('e717cb10-dc75-40d9-a823-032d7367566c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  705.809459] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ac1e112d8f26424586b016d024fb3967 in queue reply_156db6c4256a4c58884afce15e6f5340
[  705.818927] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2691167b-8e2d-4d39-9fe1-69715b7bc23b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.819141] env[60890]: Traceback (most recent call last):
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  705.819192] env[60890]:     timer()
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  705.819192] env[60890]:     cb(*args, **kw)
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  705.819192] env[60890]:     result = function(*args, **kwargs)
[  705.819192] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  705.819192] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  705.819192] env[60890]:     return method(*args, **kwargs)
[  705.819192] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  705.819192] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  705.819192] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  705.819192] env[60890]:     result = self.transport._send(
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  705.819192] env[60890]:     return self._driver.send(target, ctxt, message,
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  705.819192] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  705.819192] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  705.819192] env[60890]:     raise result
[  705.819691] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  705.819691] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  705.830357] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ac1e112d8f26424586b016d024fb3967
[  705.905226] env[60890]: INFO networking_nsxv3.common.synchronization [None req-b85c96a0-fafc-4874-8d06-190c6c16bb3c tempest-ServerActionsV293TestJSON-1045119592 tempest-ServerActionsV293TestJSON-1045119592-project-member] Enqueued Resource ID: e717cb10-dc75-40d9-a823-032d7367566c with Priority: HIGHEST for action security_group_rules
[  705.905226] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e717cb10-dc75-40d9-a823-032d7367566c with Priority: HIGHEST for action security_group_rules
[  705.905226] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.905226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.905226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  705.905226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  705.905226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.905226] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-e717cb10-dc75-40d9-a823-032d7367566c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  705.905226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  705.970217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  705.970217] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  705.970217] env[60890]: ERROR oslo_messaging.rpc.server 
[  706.059386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  706.059386] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  706.059386] env[60890]: ERROR oslo_messaging.rpc.server 
[  706.234043] env[60890]: INFO networking_nsxv3.common.synchronization [None req-a924795c-868b-45d3-8d5a-54496cc75745 tempest-AttachVolumeShelveTestJSON-2132759461 tempest-AttachVolumeShelveTestJSON-2132759461-project-member] Enqueued Resource ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f with Priority: HIGHEST for action security_group_rules
[  706.234565] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f with Priority: HIGHEST for action security_group_rules
[  706.234927] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.235060] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.235129] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.235328] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.235414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.235561] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.235657] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.235721] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.235922] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.236038] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  706.236206] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9b5c6edbd6264479b8188cb140d0bb95 in queue reply_156db6c4256a4c58884afce15e6f5340
[  706.247805] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9b5c6edbd6264479b8188cb140d0bb95
[  706.269368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.269558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.269680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.269940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.270115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.270321] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.270471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.270594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.271226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.271226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.271226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.271328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.271610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.271610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.271762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.271980] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.272159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.272255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.272455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.273377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.273616] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.273780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.273905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.274119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.274257] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  706.274485] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6cc72216dde642a0a60c47b300098407 in queue reply_156db6c4256a4c58884afce15e6f5340
[  706.281847] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6cc72216dde642a0a60c47b300098407
[  706.330612] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.330767] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.330844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.331064] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.331159] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.331256] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: e717cb10-dc75-40d9-a823-032d7367566c already deleted.
[  706.331341] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.331490] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.331651] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.331736] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('e717cb10-dc75-40d9-a823-032d7367566c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  706.331927] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 13523843743b47f0b443beb949d36256 in queue reply_156db6c4256a4c58884afce15e6f5340
[  706.345279] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 13523843743b47f0b443beb949d36256
[  706.515915] env[60890]: INFO networking_nsxv3.common.synchronization [None req-da578cf9-793f-4d16-b28c-eeeddea05c0a tempest-AttachVolumeShelveTestJSON-2132759461 tempest-AttachVolumeShelveTestJSON-2132759461-project-member] Enqueued Resource ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f with Priority: HIGHEST for action security_group_rules
[  706.516467] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f with Priority: HIGHEST for action security_group_rules
[  706.517690] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.517690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.517690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.517690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.517690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.517690] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.517690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.543286] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.543492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.543617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.543875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.544040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.544297] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.544481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.544558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.544783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.545008] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.545158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.545271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.545469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.545610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.545757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.546040] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.546192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.546308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.546544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.546689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.546870] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.547014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.547127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.547322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.547452] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  706.547685] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cbbb094c78ea4a9288269e6fd9d09c85 in queue reply_156db6c4256a4c58884afce15e6f5340
[  706.562474] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cbbb094c78ea4a9288269e6fd9d09c85
[  706.758251] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.758404] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.758476] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.758691] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.758785] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.758881] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f already deleted.
[  706.758966] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.759117] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.759276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.759360] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  706.759551] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 786d114733994d3daedac96fe7a24919 in queue reply_156db6c4256a4c58884afce15e6f5340
[  706.769476] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 786d114733994d3daedac96fe7a24919
[  706.776979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  706.777501] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  706.777501] env[60890]: ERROR oslo_messaging.rpc.server 
[  706.853305] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  706.853465] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  706.853536] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  706.853758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  706.853856] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  706.853955] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: e717cb10-dc75-40d9-a823-032d7367566c already deleted.
[  706.854683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.050657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  707.051181] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  707.051181] env[60890]: ERROR oslo_messaging.rpc.server 
[  707.057315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.057517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.057817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.058120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.058274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.058533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.058695] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.058817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.059195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.059445] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.059602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.059720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.059963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.060146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.060296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.060925] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.060925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.060925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.061088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.061148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.061366] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.061523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.061644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.061861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.061995] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  707.062224] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3045059fba724416900fb365a8a711b9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  707.074038] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3045059fba724416900fb365a8a711b9
[  707.280978] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.280978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.280978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.280978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.280978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.280978] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f already deleted.
[  707.280978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.480274] env[60890]: INFO networking_nsxv3.common.synchronization [None req-6bf10851-a722-48ef-a1dc-e504a264537a None None] Enqueued Resource ID: 591e00cf-16a1-45e0-b211-e0d2e8869869 with Priority: HIGHEST for action port
[  707.480274] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 591e00cf-16a1-45e0-b211-e0d2e8869869 with Priority: HIGHEST for action port
[  707.480274] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.480274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.480274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.480274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.480274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.480274] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-591e00cf-16a1-45e0-b211-e0d2e8869869 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.480274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-591e00cf-16a1-45e0-b211-e0d2e8869869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.480274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-591e00cf-16a1-45e0-b211-e0d2e8869869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.480274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-591e00cf-16a1-45e0-b211-e0d2e8869869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.480274] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('591e00cf-16a1-45e0-b211-e0d2e8869869',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  707.480274] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg baa913b4ddcb4aaea2bdd57105fca1f1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  707.487893] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg baa913b4ddcb4aaea2bdd57105fca1f1
[  707.564352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  707.565153] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  707.565153] env[60890]: ERROR oslo_messaging.rpc.server 
[  707.652028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.652028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.652028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.652028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.652028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.652028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  707.653325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  707.653605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  707.653947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  707.654222] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  707.654669] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 749cd96877b34c08ae9018f8f1abd52f in queue reply_156db6c4256a4c58884afce15e6f5340
[  707.660839] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 749cd96877b34c08ae9018f8f1abd52f
[  707.982785] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-591e00cf-16a1-45e0-b211-e0d2e8869869" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  707.983046] env[60890]: Traceback (most recent call last):
[  707.983077] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  707.983077] env[60890]:     timer()
[  707.983077] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  707.983077] env[60890]:     cb(*args, **kw)
[  707.983077] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  707.983077] env[60890]:     result = function(*args, **kwargs)
[  707.983215] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  707.983215] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  707.983215] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  707.983215] env[60890]:     return method(*args, **kwargs)
[  707.983215] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  707.983215] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  707.983215] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  707.983215] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  707.983215] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  707.983215] env[60890]:     result = self.transport._send(
[  707.983215] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  707.983215] env[60890]:     return self._driver.send(target, ctxt, message,
[  707.983215] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  707.983215] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  707.983215] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  707.983215] env[60890]:     raise result
[  707.983215] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  707.983215] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  708.157559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  708.158223] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  708.158223] env[60890]: ERROR oslo_messaging.rpc.server 
[  708.349627] env[60890]: INFO networking_nsxv3.common.synchronization [req-466b65bd-bd40-4c2b-88de-c5fd7ca371c1 req-7c89ccdd-3748-44b1-acb4-a3c33e999c92 tempest-ServersTestManualDisk-2117565693 tempest-ServersTestManualDisk-2117565693-project-member] Enqueued Resource ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 with Priority: HIGHEST for action security_group_members
[  708.350198] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 with Priority: HIGHEST for action security_group_members
[  708.350518] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.350643] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.350711] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.350911] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.350993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.351145] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-652ef4e3-97d8-44ab-8738-5ae16971a428 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.351237] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.351297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.351500] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.351638] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.351727] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.351852] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.351919] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.352092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.352092] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('652ef4e3-97d8-44ab-8738-5ae16971a428',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  708.352261] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d8e0a6d0fe5a40de9d5671b5158cd5f9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  708.373390] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d8e0a6d0fe5a40de9d5671b5158cd5f9
[  708.468044] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.468044] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.468044] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.468044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.468575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.468575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.468575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.468787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.468787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.469037] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.478673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.478673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.478673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.478673] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  708.478673] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 50270cc64b9c44c3a93b0a859ca7ca18 in queue reply_156db6c4256a4c58884afce15e6f5340
[  708.478673] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 50270cc64b9c44c3a93b0a859ca7ca18
[  708.853811] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.853977] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.854052] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.854268] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.854365] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.854556] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.854658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.854742] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.854896] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.854980] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.855068] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 already deleted.
[  708.855148] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.949455] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.949557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.949628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.949893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.950037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.950323] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.950575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.950643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.950815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.951059] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.951215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.951400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.951638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.951809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.951970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.952234] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.952383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.952497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.952936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.953146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.953432] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7d6a7684-0092-4751-8592-ec07e7306624 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  708.953596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  708.953718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  708.953931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  708.954066] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7d6a7684-0092-4751-8592-ec07e7306624',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  708.954300] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7fd9b9132e0f45718ee53904485ece94 in queue reply_156db6c4256a4c58884afce15e6f5340
[  708.972345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  708.972864] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  708.972864] env[60890]: ERROR oslo_messaging.rpc.server 
[  708.995883] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7fd9b9132e0f45718ee53904485ece94
[  709.181292] env[60890]: INFO networking_nsxv3.common.synchronization [None req-9779f36f-1195-4a98-a102-bcf8eeeb6be5 None None] Enqueued Resource ID: 324f77a0-badb-4e43-aa56-b800c4b680bf with Priority: HIGHEST for action port
[  709.181292] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 324f77a0-badb-4e43-aa56-b800c4b680bf with Priority: HIGHEST for action port
[  709.181577] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.181577] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.181671] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.181872] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.181957] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.182114] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-324f77a0-badb-4e43-aa56-b800c4b680bf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.182210] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-324f77a0-badb-4e43-aa56-b800c4b680bf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.182271] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-324f77a0-badb-4e43-aa56-b800c4b680bf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.182479] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-324f77a0-badb-4e43-aa56-b800c4b680bf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.182559] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('324f77a0-badb-4e43-aa56-b800c4b680bf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  709.182742] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e1a5fdd2ee6242bdab9155d5abd97ebb in queue reply_156db6c4256a4c58884afce15e6f5340
[  709.210194] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e1a5fdd2ee6242bdab9155d5abd97ebb
[  709.456662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7d6a7684-0092-4751-8592-ec07e7306624" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  709.457203] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  709.457203] env[60890]: ERROR oslo_messaging.rpc.server 
[  709.578808] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.578808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.578808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.578808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.579013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.579205] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.579357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.579473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.579689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.579970] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.580156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.580273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.580483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.580623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.580772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.581018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.581168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.581285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.581499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.581635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.581875] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.582034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.582156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.582805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.582805] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  709.582805] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg eca4e6f9f477456f8fcfaa69895d8b30 in queue reply_156db6c4256a4c58884afce15e6f5340
[  709.597214] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eca4e6f9f477456f8fcfaa69895d8b30
[  709.684667] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-324f77a0-badb-4e43-aa56-b800c4b680bf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.684972] env[60890]: Traceback (most recent call last):
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  709.685031] env[60890]:     timer()
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  709.685031] env[60890]:     cb(*args, **kw)
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  709.685031] env[60890]:     result = function(*args, **kwargs)
[  709.685031] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  709.685031] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  709.685031] env[60890]:     return method(*args, **kwargs)
[  709.685031] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  709.685031] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  709.685031] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  709.685031] env[60890]:     result = self.transport._send(
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  709.685031] env[60890]:     return self._driver.send(target, ctxt, message,
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  709.685031] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  709.685031] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  709.685031] env[60890]:     raise result
[  709.685670] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  709.685670] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  709.920426] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.920644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.920774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.921048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.921197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.921465] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.921621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.921742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.922028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.922278] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.922435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.922556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.922794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.922951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.923109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.923373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.923532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.923652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.923862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.923997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  709.924535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  709.924701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  709.924903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  709.925169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  709.925310] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  709.925552] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0eeb2a4140f8453283ad49db3bed2dd8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  709.932341] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0eeb2a4140f8453283ad49db3bed2dd8
[  710.087310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  710.087849] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  710.087849] env[60890]: ERROR oslo_messaging.rpc.server 
[  710.428251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  710.428778] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  710.428778] env[60890]: ERROR oslo_messaging.rpc.server 
[  710.611156] env[60890]: INFO networking_nsxv3.common.synchronization [req-aac9b989-cb2a-487f-9f8f-6df3dcc8da66 req-67257cc5-8736-4e1a-b8fb-48e95d9c5ef9 tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 with Priority: HIGHEST for action security_group_members
[  710.611550] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 with Priority: HIGHEST for action security_group_members
[  710.611874] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.612482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.612561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.612767] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.612856] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.613023] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.613124] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.613188] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.613339] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.613715] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.613833] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.613914] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.614041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.614122] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.614213] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  710.614394] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3167067dc695485cb30a373e435956f9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  710.622774] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3167067dc695485cb30a373e435956f9
[  710.702923] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.703139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.703273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.703541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.703691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.706976] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.706976] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.706976] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.706976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.707709] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.708238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.708238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.708335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.708593] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  710.708941] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 28b1b555df6c4c01966982d23ec5f0b1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  710.715438] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 28b1b555df6c4c01966982d23ec5f0b1
[  710.989043] env[60890]: INFO networking_nsxv3.common.synchronization [None req-65203c31-2d36-493e-8555-d33804e23d81 None None] Enqueued Resource ID: 2f55c9f4-5ca1-420e-ad69-5ab3d1d06ef9 with Priority: HIGHEST for action port
[  710.989043] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2f55c9f4-5ca1-420e-ad69-5ab3d1d06ef9 with Priority: HIGHEST for action port
[  710.989043] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.989043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.989043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.989043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.989043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  710.989043] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2f55c9f4-5ca1-420e-ad69-5ab3d1d06ef9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  710.989043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2f55c9f4-5ca1-420e-ad69-5ab3d1d06ef9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  710.989043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2f55c9f4-5ca1-420e-ad69-5ab3d1d06ef9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  710.989043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2f55c9f4-5ca1-420e-ad69-5ab3d1d06ef9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  710.989043] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2f55c9f4-5ca1-420e-ad69-5ab3d1d06ef9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  710.989043] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 70068bbdb5694ad0b21a4122ebcc0228 in queue reply_156db6c4256a4c58884afce15e6f5340
[  710.993757] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 70068bbdb5694ad0b21a4122ebcc0228
[  711.122513] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.122513] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.122513] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88 already deleted.
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c8ba299c-d9b6-4dfa-8d0b-671eeedd9f88" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.122513] env[60890]: INFO networking_nsxv3.common.synchronization [req-f947cd28-004c-4c02-82a5-95f42ef37b48 req-8ffe0b15-2c37-4b42-8910-3b1c7f80401a tempest-ImagesOneServerTestJSON-29343680 tempest-ImagesOneServerTestJSON-29343680-project-member] Enqueued Resource ID: 17fbbde3-5636-48b6-871a-8128afdf1e9b with Priority: HIGHEST for action security_group_members
[  711.122513] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 17fbbde3-5636-48b6-871a-8128afdf1e9b with Priority: HIGHEST for action security_group_members
[  711.122513] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.122513] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-17fbbde3-5636-48b6-871a-8128afdf1e9b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-17fbbde3-5636-48b6-871a-8128afdf1e9b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-17fbbde3-5636-48b6-871a-8128afdf1e9b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-17fbbde3-5636-48b6-871a-8128afdf1e9b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.122513] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.122513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.122513] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('17fbbde3-5636-48b6-871a-8128afdf1e9b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  711.122513] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7292e9be52294ea89231c9e3176db015 in queue reply_156db6c4256a4c58884afce15e6f5340
[  711.151303] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7292e9be52294ea89231c9e3176db015
[  711.213972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  711.213972] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  711.213972] env[60890]: ERROR oslo_messaging.rpc.server 
[  711.480387] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.480387] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.480387] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.480387] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.480387] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.480387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.482543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.482543] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  711.483041] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1b6573b9437b40be9a1cbfe75e9630c7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  711.489509] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1b6573b9437b40be9a1cbfe75e9630c7
[  711.491058] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2f55c9f4-5ca1-420e-ad69-5ab3d1d06ef9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.491227] env[60890]: Traceback (most recent call last):
[  711.491263] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  711.491263] env[60890]:     timer()
[  711.491263] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  711.491263] env[60890]:     cb(*args, **kw)
[  711.491263] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  711.491263] env[60890]:     result = function(*args, **kwargs)
[  711.491263] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  711.491263] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  711.491263] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  711.491263] env[60890]:     return method(*args, **kwargs)
[  711.491263] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  711.491263] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  711.491263] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  711.491263] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  711.491610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  711.491610] env[60890]:     result = self.transport._send(
[  711.491610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  711.491610] env[60890]:     return self._driver.send(target, ctxt, message,
[  711.491610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  711.491610] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  711.491610] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  711.491610] env[60890]:     raise result
[  711.491610] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  711.491610] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  711.530036] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.530236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.530361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.530927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.530927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.531227] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.531227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.531310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.531564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.531763] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.531911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.533584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.533899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.534149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.534218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.534550] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.534726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.534871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.535095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.535237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.535486] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.535645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.535770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.535991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.536193] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  711.536446] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 636c46898bc542de9264f06f6d014f05 in queue reply_156db6c4256a4c58884afce15e6f5340
[  711.558888] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 636c46898bc542de9264f06f6d014f05
[  711.624547] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.624721] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.624810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.625033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.625128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.625327] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.625619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.625619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.625619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.625738] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.625785] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 17fbbde3-5636-48b6-871a-8128afdf1e9b already deleted.
[  711.625864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-17fbbde3-5636-48b6-871a-8128afdf1e9b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.637968] env[60890]: INFO networking_nsxv3.common.synchronization [None req-a2eca1e0-90ff-49d8-bbd2-3d91db767a0d None None] Enqueued Resource ID: 2ccd28dc-cbbd-40b2-bff5-57b3de9a792d with Priority: HIGHEST for action port
[  711.640012] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2ccd28dc-cbbd-40b2-bff5-57b3de9a792d with Priority: HIGHEST for action port
[  711.640012] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.640012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.640012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.640012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.640012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.640012] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2ccd28dc-cbbd-40b2-bff5-57b3de9a792d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  711.640012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2ccd28dc-cbbd-40b2-bff5-57b3de9a792d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  711.640012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2ccd28dc-cbbd-40b2-bff5-57b3de9a792d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  711.640012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2ccd28dc-cbbd-40b2-bff5-57b3de9a792d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  711.640012] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2ccd28dc-cbbd-40b2-bff5-57b3de9a792d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  711.640012] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 13f3730f91314da8b467db726f053108 in queue reply_156db6c4256a4c58884afce15e6f5340
[  711.646247] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 13f3730f91314da8b467db726f053108
[  711.986318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  711.986899] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  711.986899] env[60890]: ERROR oslo_messaging.rpc.server 
[  712.041089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  712.041598] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  712.041598] env[60890]: ERROR oslo_messaging.rpc.server 
[  712.141897] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2ccd28dc-cbbd-40b2-bff5-57b3de9a792d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.142222] env[60890]: Traceback (most recent call last):
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  712.142295] env[60890]:     timer()
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  712.142295] env[60890]:     cb(*args, **kw)
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  712.142295] env[60890]:     result = function(*args, **kwargs)
[  712.142295] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  712.142295] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  712.142295] env[60890]:     return method(*args, **kwargs)
[  712.142295] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  712.142295] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  712.142295] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  712.142295] env[60890]:     result = self.transport._send(
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  712.142295] env[60890]:     return self._driver.send(target, ctxt, message,
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  712.142295] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  712.142295] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  712.142295] env[60890]:     raise result
[  712.142847] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  712.142847] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  712.656865] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.657183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.657249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.657525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.657671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.658010] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.658103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.658214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.658431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.658671] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.658826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.658964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.659356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.659356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.659451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.660294] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.660294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.660294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.660294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.660294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.660533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-181946f8-2475-4d03-8ae6-0fdb31c3782a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.660753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.660753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.660964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.661157] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('181946f8-2475-4d03-8ae6-0fdb31c3782a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  712.661337] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5460c29eb2ec41299248cae787db6082 in queue reply_156db6c4256a4c58884afce15e6f5340
[  712.666676] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5460c29eb2ec41299248cae787db6082
[  712.747841] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.748102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.748205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.748471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.748616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.748886] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.749045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.749166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.749431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.749738] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.749843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.749957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.750246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.750397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.750666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.750803] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.750964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.751085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.751296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.751571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.751692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.751828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.751949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.752237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.752616] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  712.752690] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7d4fab1031c045de9415d08a68c5e0ce in queue reply_156db6c4256a4c58884afce15e6f5340
[  712.754506] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.754675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.754886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.755116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.755261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.755508] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.755665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.755813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.756033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.756864] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.756864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.756864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.756967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.757101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.757171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.757922] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.758290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.758290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.758443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.758582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  712.759407] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  712.759407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  712.759407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  712.759407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  712.759407] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  712.759645] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 46cb0488ac3940b7985eaf970b06c87e in queue reply_156db6c4256a4c58884afce15e6f5340
[  712.760656] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7d4fab1031c045de9415d08a68c5e0ce
[  712.763478] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 46cb0488ac3940b7985eaf970b06c87e
[  713.163674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-181946f8-2475-4d03-8ae6-0fdb31c3782a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  713.164226] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  713.164226] env[60890]: ERROR oslo_messaging.rpc.server 
[  713.261697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  713.261916] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  713.261916] env[60890]: ERROR oslo_messaging.rpc.server 
[  713.263159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  713.263574] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  713.263574] env[60890]: ERROR oslo_messaging.rpc.server 
[  713.548906] env[60890]: INFO networking_nsxv3.common.synchronization [None req-f1eeda06-bb3d-455d-ba66-440bcb1dae6c None None] Enqueued Resource ID: 8a652ccd-1cbd-4fb2-8f04-70305e9597d5 with Priority: HIGHEST for action port
[  713.549154] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8a652ccd-1cbd-4fb2-8f04-70305e9597d5 with Priority: HIGHEST for action port
[  713.549770] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  713.549888] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  713.549957] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  713.550157] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  713.550301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  713.550447] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-8a652ccd-1cbd-4fb2-8f04-70305e9597d5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  713.550606] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-8a652ccd-1cbd-4fb2-8f04-70305e9597d5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  713.550702] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-8a652ccd-1cbd-4fb2-8f04-70305e9597d5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  713.550915] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-8a652ccd-1cbd-4fb2-8f04-70305e9597d5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  713.550993] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('8a652ccd-1cbd-4fb2-8f04-70305e9597d5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  713.551171] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9820a290b5bb49349ce1b0d5d2f026fb in queue reply_156db6c4256a4c58884afce15e6f5340
[  713.565447] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9820a290b5bb49349ce1b0d5d2f026fb
[  713.965591] env[60890]: INFO networking_nsxv3.common.synchronization [req-d2f0426c-c76d-4fd2-8549-ae020bed1361 req-217edd78-a314-4181-a6f4-012f58253b22 tempest-AttachVolumeTestJSON-389769224 tempest-AttachVolumeTestJSON-389769224-project-member] Enqueued Resource ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 with Priority: HIGHEST for action security_group_members
[  713.966145] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 with Priority: HIGHEST for action security_group_members
[  713.966457] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  713.966577] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  713.966642] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  713.966831] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  713.966910] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  713.967069] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-73956ef6-bfe0-49e9-80aa-feabeac5ab67 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  713.967161] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  713.967220] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  713.967365] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  713.967499] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  713.967584] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  713.967642] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  713.967775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  713.967875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  713.967965] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('73956ef6-bfe0-49e9-80aa-feabeac5ab67',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  713.968163] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8a548f86752e40129c292532a133c3b9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  713.980501] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8a548f86752e40129c292532a133c3b9
[  714.030812] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.031014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.031200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.031402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.031550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.031806] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.031964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.032101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.032335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.032553] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.039132] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.039132] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.039132] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  714.039132] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg be10318b46d0455ea8b76f90449b2ae8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  714.039132] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.039132] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.039132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.040248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.040248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.040248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.040248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.040248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.040248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.041350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.041350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.041350] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.041350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.041350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.041350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.041673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.041884] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.042042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.042216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.042367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.042503] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  714.042739] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f66d3c67abec42c0bc48cea02e3d08a9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  714.044119] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg be10318b46d0455ea8b76f90449b2ae8
[  714.047106] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f66d3c67abec42c0bc48cea02e3d08a9
[  714.052632] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-8a652ccd-1cbd-4fb2-8f04-70305e9597d5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.052884] env[60890]: Traceback (most recent call last):
[  714.052919] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  714.052919] env[60890]:     timer()
[  714.052919] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  714.052919] env[60890]:     cb(*args, **kw)
[  714.052919] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  714.052919] env[60890]:     result = function(*args, **kwargs)
[  714.052919] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  714.052919] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  714.052919] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  714.052919] env[60890]:     return method(*args, **kwargs)
[  714.052919] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  714.052919] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  714.052919] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  714.052919] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  714.053256] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  714.053256] env[60890]:     result = self.transport._send(
[  714.053256] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  714.053256] env[60890]:     return self._driver.send(target, ctxt, message,
[  714.053256] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  714.053256] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  714.053256] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  714.053256] env[60890]:     raise result
[  714.053256] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  714.053256] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  714.458957] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9e8fb3b0f7a94e179137c2ebcd9f95d5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  714.472378] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9e8fb3b0f7a94e179137c2ebcd9f95d5
[  714.473392] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.473524] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.473650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.473803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.473896] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.474081] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  714.474221] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  714.474321] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  714.474520] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  714.474561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.474648] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 73956ef6-bfe0-49e9-80aa-feabeac5ab67 already deleted.
[  714.474776] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-73956ef6-bfe0-49e9-80aa-feabeac5ab67" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.546832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  714.549814] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  714.549814] env[60890]: ERROR oslo_messaging.rpc.server 
[  714.549814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  714.550768] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  714.550768] env[60890]: ERROR oslo_messaging.rpc.server 
[  714.606033] env[60890]: INFO neutron.plugins.ml2.drivers.agent._common_agent [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] NSXv3 Agent Agent loop - iteration:0 completed
[  714.606242] env[60890]: INFO neutron.plugins.ml2.drivers.agent._common_agent [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] NSXv3 Agent Agent loop - iteration:1 started
[  715.473204] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.473204] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.473204] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.473204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.473867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.473867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.473867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.474106] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.474106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.474311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.474793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.474793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.474793] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.474975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.482765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.483069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.483226] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  715.483456] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 40baf4a4e03e4d0097c859eda8f06e1a in queue reply_156db6c4256a4c58884afce15e6f5340
[  715.485525] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.485691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.485840] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.486064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.486200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.486444] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.486593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.486711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.486968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.487326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.487326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.487443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.487611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.487760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.487907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.488172] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.488321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.488437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.488640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.488767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.489001] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  715.489144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  715.489258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  715.489458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  715.489583] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  715.489930] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ce878217f3844e509d4f9b783ddbec7e in queue reply_156db6c4256a4c58884afce15e6f5340
[  715.495729] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 40baf4a4e03e4d0097c859eda8f06e1a
[  715.499320] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ce878217f3844e509d4f9b783ddbec7e
[  715.995517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  715.996116] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  715.996116] env[60890]: ERROR oslo_messaging.rpc.server 
[  715.997273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  715.997676] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  715.997676] env[60890]: ERROR oslo_messaging.rpc.server 
[  716.688128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  716.688128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.688128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  716.688128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  716.688128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.688128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.688128] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  716.689909] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0534bf5246fd48e1bf5415829dfbe3fa in queue reply_156db6c4256a4c58884afce15e6f5340
[  716.721607] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.721817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.721946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.722210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.722356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  716.722617] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.722794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.722925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.723144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.723388] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.723544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.723662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.723874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.724058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  716.724210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  716.724461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.724617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.724755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.724980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.725118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  716.725361] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  716.725516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  716.725635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  716.725852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  716.725988] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  716.726226] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 65765273107f4cfe8fe432610baa6993 in queue reply_156db6c4256a4c58884afce15e6f5340
[  716.779429] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0534bf5246fd48e1bf5415829dfbe3fa
[  716.780326] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 65765273107f4cfe8fe432610baa6993
[  717.186757] env[60890]: INFO networking_nsxv3.common.synchronization [None req-3ec8f0e2-483e-4421-8fea-82bd907d841f None None] Enqueued Resource ID: b38b1b15-a0a9-4d1f-b729-faf36d432af1 with Priority: HIGHEST for action port
[  717.187018] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b38b1b15-a0a9-4d1f-b729-faf36d432af1 with Priority: HIGHEST for action port
[  717.187614] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.187738] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.187866] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.188103] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.188197] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.188380] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-b38b1b15-a0a9-4d1f-b729-faf36d432af1 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.188481] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-b38b1b15-a0a9-4d1f-b729-faf36d432af1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.188544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-b38b1b15-a0a9-4d1f-b729-faf36d432af1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.188740] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-b38b1b15-a0a9-4d1f-b729-faf36d432af1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.188826] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b38b1b15-a0a9-4d1f-b729-faf36d432af1',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  717.189005] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg bfaa46d4102c43a6a28364d2913236d4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  717.192222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  717.193051] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  717.193051] env[60890]: ERROR oslo_messaging.rpc.server 
[  717.200472] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bfaa46d4102c43a6a28364d2913236d4
[  717.228328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  717.228854] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  717.228854] env[60890]: ERROR oslo_messaging.rpc.server 
[  717.286893] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.287137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.287673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.287970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.288151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.288421] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.288585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.288709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.289012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.289279] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.290101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.291597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.291886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.292111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.292337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.292652] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.292927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.293123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.293453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.293726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.294170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.294479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.294778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.295054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.295237] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  717.295515] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7b4abcd04fe04093a4eb79a83d48e771 in queue reply_156db6c4256a4c58884afce15e6f5340
[  717.304942] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7b4abcd04fe04093a4eb79a83d48e771
[  717.695161] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-b38b1b15-a0a9-4d1f-b729-faf36d432af1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.695543] env[60890]: Traceback (most recent call last):
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  717.695543] env[60890]:     timer()
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  717.695543] env[60890]:     cb(*args, **kw)
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  717.695543] env[60890]:     result = function(*args, **kwargs)
[  717.695543] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  717.695543] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  717.695543] env[60890]:     return method(*args, **kwargs)
[  717.695543] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  717.695543] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  717.695543] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  717.695543] env[60890]:     result = self.transport._send(
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  717.695543] env[60890]:     return self._driver.send(target, ctxt, message,
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  717.695543] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  717.695543] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  717.695543] env[60890]:     raise result
[  717.695543] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  717.695543] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  717.797895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  717.798417] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  717.798417] env[60890]: ERROR oslo_messaging.rpc.server 
[  717.843336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.843545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.843675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.844083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.844261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.844523] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.844680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.844884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.845116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.845360] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.845510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.845625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.845878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.846021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.846169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.846420] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.846568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.846683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.846942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.847009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  717.847237] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  717.847383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  717.847495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  717.847716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  717.847819] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  717.848093] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e86af349c59e4e74a7c91fc9c8e551d5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  717.854288] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e86af349c59e4e74a7c91fc9c8e551d5
[  718.351461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  718.351461] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  718.351461] env[60890]: ERROR oslo_messaging.rpc.server 
[  718.438878] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.439084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.439222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.439513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.439660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.439998] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.440341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.440341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.440492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.440684] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.440853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.440974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.441180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.441318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.441467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.441731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.441889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.442007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.442259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.442393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.442638] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.442872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.442920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.443124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.443236] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  718.443469] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 38441f2f39594fa380d86549fd475867 in queue reply_156db6c4256a4c58884afce15e6f5340
[  718.449347] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 38441f2f39594fa380d86549fd475867
[  718.922589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.922589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.922589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.922589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.922589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  718.922589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  718.922589] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  718.922589] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a1b755be8bf540e89fa21064f25ddbe5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  718.929988] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a1b755be8bf540e89fa21064f25ddbe5
[  718.946358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  718.947055] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  718.947055] env[60890]: ERROR oslo_messaging.rpc.server 
[  719.426441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  719.427766] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  719.427766] env[60890]: ERROR oslo_messaging.rpc.server 
[  719.517380] env[60890]: INFO networking_nsxv3.common.synchronization [None req-dabb0a5c-3e16-4098-b884-841568d59f61 None None] Enqueued Resource ID: f9718120-0b7d-4752-a05d-e7b8b425d66b with Priority: HIGHEST for action port
[  719.518942] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f9718120-0b7d-4752-a05d-e7b8b425d66b with Priority: HIGHEST for action port
[  719.519745] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.519999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.520156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.520427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.520577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.520860] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.521000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.521122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.521350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.521594] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.521755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.521890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.522083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.522329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.522400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.522646] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.522944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.523102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.523337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.523484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.523912] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.524064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.524200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.524423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.524566] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  719.524861] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2044c8121ad74ac1ac496f0d6fe7724c in queue reply_156db6c4256a4c58884afce15e6f5340
[  719.525683] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.525794] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.525927] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.526011] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.526078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.526217] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-f9718120-0b7d-4752-a05d-e7b8b425d66b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.526306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-f9718120-0b7d-4752-a05d-e7b8b425d66b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.526412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-f9718120-0b7d-4752-a05d-e7b8b425d66b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.526544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-f9718120-0b7d-4752-a05d-e7b8b425d66b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.526617] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f9718120-0b7d-4752-a05d-e7b8b425d66b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  719.526780] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3b664609506b405c9a35b7336b5880dc in queue reply_156db6c4256a4c58884afce15e6f5340
[  719.532031] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2044c8121ad74ac1ac496f0d6fe7724c
[  719.534328] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3b664609506b405c9a35b7336b5880dc
[  719.969088] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.969377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.969412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.969686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.969836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.970098] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.970253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.970370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.970599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.970836] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.970991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.971109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.971314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.971452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.971600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.971848] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.971998] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.972130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.972340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.972472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  719.972757] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  719.972917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  719.973034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  719.973246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  719.973378] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  719.973607] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fb2818787ec5411390766e6a6e996da8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  719.981611] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fb2818787ec5411390766e6a6e996da8
[  720.028699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  720.029197] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  720.029197] env[60890]: ERROR oslo_messaging.rpc.server 
[  720.030492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-f9718120-0b7d-4752-a05d-e7b8b425d66b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.030492] env[60890]: Traceback (most recent call last):
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  720.030492] env[60890]:     timer()
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  720.030492] env[60890]:     cb(*args, **kw)
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  720.030492] env[60890]:     result = function(*args, **kwargs)
[  720.030492] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  720.030492] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  720.030492] env[60890]:     return method(*args, **kwargs)
[  720.030492] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  720.030492] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  720.030492] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  720.030492] env[60890]:     result = self.transport._send(
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  720.030492] env[60890]:     return self._driver.send(target, ctxt, message,
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  720.030492] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  720.030492] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  720.030492] env[60890]:     raise result
[  720.030492] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  720.030492] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  720.476429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  720.476745] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  720.476745] env[60890]: ERROR oslo_messaging.rpc.server 
[  720.537053] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.538215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.538215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.538215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.538215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.538215] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-347 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.538474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.538474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.538582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.538907] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.539039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.539125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.539343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.539496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.539726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-347" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.540030] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.540217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.540338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.540551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.540693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.540940] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-daa8aba5-8f7a-4066-9a48-d4e76b738e71 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.541105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.541254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.541475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.541651] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('daa8aba5-8f7a-4066-9a48-d4e76b738e71',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  720.541960] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c1192d1d87eb4dec9a69a2b0dfa7bfcd in queue reply_156db6c4256a4c58884afce15e6f5340
[  720.547216] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c1192d1d87eb4dec9a69a2b0dfa7bfcd
[  720.583595] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.583804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.583937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.584234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.584384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.584642] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.584861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.584986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.585209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.585839] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.585839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.585839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.586112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.586112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.586177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.586424] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.586576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.586695] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.586901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.587034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  720.587286] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  720.587441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  720.587557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  720.587779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  720.587889] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  720.588136] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2713548bda8f46338ef40db5c8df44dd in queue reply_156db6c4256a4c58884afce15e6f5340
[  720.594219] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2713548bda8f46338ef40db5c8df44dd
[  721.044419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-daa8aba5-8f7a-4066-9a48-d4e76b738e71" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  721.045209] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  721.045209] env[60890]: ERROR oslo_messaging.rpc.server 
[  721.090158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  721.090771] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  721.090771] env[60890]: ERROR oslo_messaging.rpc.server 
[  721.852801] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.852801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.852801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.852992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.853129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.853394] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.853549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.853669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.853887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.854131] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.854283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.854402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.854612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.854778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.854935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.855201] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.855369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.855488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.855733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.855862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.856080] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.856239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.856356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.856564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.856701] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  721.856944] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fc5706af05634392b7a8cd9cce61810f in queue reply_156db6c4256a4c58884afce15e6f5340
[  721.863359] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fc5706af05634392b7a8cd9cce61810f
[  721.971395] env[60890]: INFO networking_nsxv3.common.synchronization [req-466b65bd-bd40-4c2b-88de-c5fd7ca371c1 req-d566bf3b-bcb9-44a7-926a-e78bdd54a1bd tempest-ServersTestManualDisk-2117565693 tempest-ServersTestManualDisk-2117565693-project-member] Enqueued Resource ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 with Priority: HIGHEST for action security_group_members
[  721.971998] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 with Priority: HIGHEST for action security_group_members
[  721.972298] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.972465] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.972574] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.977161] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.977275] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.977449] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-652ef4e3-97d8-44ab-8738-5ae16971a428 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.977561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.977623] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.977784] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.977937] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  721.978032] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  721.978097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  721.978242] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  721.978326] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  721.978421] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('652ef4e3-97d8-44ab-8738-5ae16971a428',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  721.978606] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 444a272512444a669ef0f3396fa94245 in queue reply_156db6c4256a4c58884afce15e6f5340
[  721.986706] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 444a272512444a669ef0f3396fa94245
[  722.361350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  722.361350] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  722.361350] env[60890]: ERROR oslo_messaging.rpc.server 
[  722.480359] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  722.480525] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  722.480600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  722.480826] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  722.480928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  722.481122] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  722.481228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  722.481293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  722.481453] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  722.481538] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  722.481625] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 652ef4e3-97d8-44ab-8738-5ae16971a428 already deleted.
[  722.481705] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-652ef4e3-97d8-44ab-8738-5ae16971a428" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  722.933240] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  722.933436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  722.933563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  722.933845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  722.933990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  722.934242] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-222 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  722.934395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  722.934514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  722.934775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  722.935049] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  722.935204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  722.935322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  722.935558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  722.935702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  722.935866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-222" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  722.936153] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  722.936306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  722.936423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  722.936634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  722.936765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  722.937001] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  722.937152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  722.937267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  722.937489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  722.937623] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('80a7af89-4e51-4fab-9a57-bb4998bf5ad3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  722.937852] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c8f50a32f7274d5f9ebb7decfae5fb70 in queue reply_156db6c4256a4c58884afce15e6f5340
[  722.943279] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c8f50a32f7274d5f9ebb7decfae5fb70
[  723.440182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-80a7af89-4e51-4fab-9a57-bb4998bf5ad3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  723.440182] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  723.440182] env[60890]: ERROR oslo_messaging.rpc.server 
[  724.846540] env[60890]: INFO networking_nsxv3.common.synchronization [req-f947cd28-004c-4c02-82a5-95f42ef37b48 req-de79f5c6-d4a4-4137-b50b-c59c89c73742 tempest-ImagesOneServerTestJSON-29343680 tempest-ImagesOneServerTestJSON-29343680-project-member] Enqueued Resource ID: 17fbbde3-5636-48b6-871a-8128afdf1e9b with Priority: HIGHEST for action security_group_members
[  724.846540] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 17fbbde3-5636-48b6-871a-8128afdf1e9b with Priority: HIGHEST for action security_group_members
[  724.846540] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  724.846540] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-17fbbde3-5636-48b6-871a-8128afdf1e9b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-17fbbde3-5636-48b6-871a-8128afdf1e9b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-17fbbde3-5636-48b6-871a-8128afdf1e9b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-17fbbde3-5636-48b6-871a-8128afdf1e9b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  724.846540] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  724.846540] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  724.846540] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('17fbbde3-5636-48b6-871a-8128afdf1e9b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  724.846540] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 981fc2a9e73a407bbe5bed4376fcd43d in queue reply_156db6c4256a4c58884afce15e6f5340
[  724.859696] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 981fc2a9e73a407bbe5bed4376fcd43d
[  725.351157] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  725.351363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  725.351438] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  725.351691] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  725.351792] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  725.352017] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  725.352181] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  725.352219] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  725.352346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  725.352426] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  725.352515] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 17fbbde3-5636-48b6-871a-8128afdf1e9b already deleted.
[  725.352613] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-17fbbde3-5636-48b6-871a-8128afdf1e9b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  729.844048] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  729.844319] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  731.018763] env[60890]: INFO networking_nsxv3.common.synchronization [req-46b5533c-3eb5-4b30-9b6e-9a68a1ec4ad1 req-d078e776-8326-4b03-97b4-a51cb5692185 tempest-InstanceActionsTestJSON-1548267367 tempest-InstanceActionsTestJSON-1548267367-project-member] Enqueued Resource ID: c4b2c93a-cefc-4fba-89f9-5076b85229e7 with Priority: HIGHEST for action security_group_members
[  731.018763] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c4b2c93a-cefc-4fba-89f9-5076b85229e7 with Priority: HIGHEST for action security_group_members
[  731.018763] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.018763] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c4b2c93a-cefc-4fba-89f9-5076b85229e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c4b2c93a-cefc-4fba-89f9-5076b85229e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c4b2c93a-cefc-4fba-89f9-5076b85229e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c4b2c93a-cefc-4fba-89f9-5076b85229e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.018763] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.018763] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.018763] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c4b2c93a-cefc-4fba-89f9-5076b85229e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  731.018763] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b536d3d9988743d3a5fe5be10980aa9a in queue reply_156db6c4256a4c58884afce15e6f5340
[  731.032759] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b536d3d9988743d3a5fe5be10980aa9a
[  731.526897] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.527136] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.527186] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.527611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.527611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.527819] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.527938] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.528048] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.528217] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.528297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.528383] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c4b2c93a-cefc-4fba-89f9-5076b85229e7 already deleted.
[  731.528496] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c4b2c93a-cefc-4fba-89f9-5076b85229e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.581764] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.581977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.582110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.582462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.582676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.582984] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.583156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.583280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.583648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.583951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.584141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.584316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.584579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.584755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.584928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.585207] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.585392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.585556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.585819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.586008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  731.586297] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  731.586510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  731.586672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  731.586938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  731.587153] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  731.587467] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e82cc5a76c3349a68595c8f8c336c7e1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  731.593323] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e82cc5a76c3349a68595c8f8c336c7e1
[  732.089548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  732.090048] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  732.090048] env[60890]: ERROR oslo_messaging.rpc.server 
[  732.699533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  732.699533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  732.699698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  732.699882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  732.700062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  732.700333] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  732.700523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  732.700648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  732.700943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  732.701201] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  732.701344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  732.701454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  732.701669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  732.701821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  732.702022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  732.702350] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  732.702557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  732.702871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  732.702983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  732.703071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  732.703316] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  732.703507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  732.703634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  732.703847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  732.703982] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  732.704238] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1e398a7105944f65baf7a7423fe14e64 in queue reply_156db6c4256a4c58884afce15e6f5340
[  732.709500] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1e398a7105944f65baf7a7423fe14e64
[  733.206596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  733.207140] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  733.207140] env[60890]: ERROR oslo_messaging.rpc.server 
[  733.769939] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  733.770153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  733.770297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  733.770605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  733.770767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  733.771031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  733.771188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  733.771309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  733.771538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  733.771787] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  733.771960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  733.772159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  733.772350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  733.772508] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  733.772641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  733.772882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  733.773032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  733.773147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  733.773356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  733.773509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  733.773756] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  733.773907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  733.774024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  733.774229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  733.774357] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  733.774588] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 11f506526d4844f2b6cf5df887a702a3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  733.780320] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 11f506526d4844f2b6cf5df887a702a3
[  734.277268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  734.277847] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  734.277847] env[60890]: ERROR oslo_messaging.rpc.server 
[  734.851154] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  734.851370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  734.851504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  734.851786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  734.851940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  734.852229] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  734.852395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  734.852519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  734.852760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  734.853501] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  734.853501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  734.853501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  734.853501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  734.853760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  734.853805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  734.854045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  734.854206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  734.854329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  734.854544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  734.854685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  734.854978] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  734.855138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  734.855261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  734.855474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  734.855613] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  734.855870] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2fdf7114f9d84da291dea04ad3ab6638 in queue reply_156db6c4256a4c58884afce15e6f5340
[  734.861135] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2fdf7114f9d84da291dea04ad3ab6638
[  735.358423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  735.358836] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  735.358836] env[60890]: ERROR oslo_messaging.rpc.server 
[  735.951575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  735.951854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  735.951934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  735.952724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  735.952724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  735.952842] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  735.952965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  735.953142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  735.953427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  735.953750] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  735.954120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  735.954258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  735.954526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  735.955797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  735.955797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  735.955797] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  735.955797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  735.955797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  735.955797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  735.955797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  735.956181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  735.956181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  735.956258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  735.956498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  735.956641] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  735.956873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ef8f4b4b5d80447f87ddac57884b81e6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  735.962016] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ef8f4b4b5d80447f87ddac57884b81e6
[  736.006546] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  736.006647] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  736.459377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  736.459983] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  736.459983] env[60890]: ERROR oslo_messaging.rpc.server 
[  737.007632] env[60890]: INFO networking_nsxv3.common.synchronization [None req-09d37ebe-c540-4391-9f23-f2e5363d38eb tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 with Priority: HIGHEST for action security_group_rules
[  737.008181] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 with Priority: HIGHEST for action security_group_rules
[  737.008501] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.008627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.008692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.008895] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.008984] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.009186] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.009244] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.009305] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.009515] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.009595] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('f6f508fa-9e0c-4458-b197-ea1aab3b5c03',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  737.009775] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2437ab3d607b451f8f5090200f44d75d in queue reply_156db6c4256a4c58884afce15e6f5340
[  737.019357] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2437ab3d607b451f8f5090200f44d75d
[  737.150481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.150593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.150718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.150982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.151120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.151368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.151512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.151624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.151837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.152089] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.152237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.152349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.152548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.152782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.152868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.153094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.153240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.153352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.153552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.153679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.153907] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.154053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.154167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.154367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.154496] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  737.154817] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a4c94f4d274e4cc28601379b47ff3484 in queue reply_156db6c4256a4c58884afce15e6f5340
[  737.162857] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a4c94f4d274e4cc28601379b47ff3484
[  737.164197] env[60890]: INFO networking_nsxv3.common.synchronization [req-7e62106b-acdb-40c5-aba3-2bbc6dc831dc req-db689777-4bf8-449e-b348-4b4784c7709a tempest-MultipleCreateTestJSON-1912904963 tempest-MultipleCreateTestJSON-1912904963-project-member] Enqueued Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  737.164197] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  737.164327] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.164437] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.164500] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.164676] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.164788] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.165118] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-08b77cbe-1697-46da-85d2-54fa8e8cc834 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.165118] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.165118] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.165280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.165404] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.165492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.165550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.165688] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.165765] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.165851] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('08b77cbe-1697-46da-85d2-54fa8e8cc834',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  737.166021] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7c41489ac2e140c184a010daafe3d9d5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  737.178760] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7c41489ac2e140c184a010daafe3d9d5
[  737.290266] env[60890]: INFO networking_nsxv3.common.synchronization [None req-dbbb49a8-b5b1-46c4-bcc2-c5ee182b9d56 tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 with Priority: HIGHEST for action security_group_rules
[  737.291045] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 with Priority: HIGHEST for action security_group_rules
[  737.291385] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.291515] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.291592] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.291837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.291932] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.292116] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.292202] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.507693] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.508017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.508060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.508304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.508454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.508716] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.508879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.509005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.509596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.509891] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.510071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.510198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.510432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.510594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.510752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.511033] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.511193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.511332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.511560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.511703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.511971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.512147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.512282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.512547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.512699] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  737.512992] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 488924870c0346e6a4f6f4c438efaac9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  737.518390] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 488924870c0346e6a4f6f4c438efaac9
[  737.531118] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.531250] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.531297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.531478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.531574] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.531667] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 already deleted.
[  737.531752] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.531905] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.532085] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.532172] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('f6f508fa-9e0c-4458-b197-ea1aab3b5c03',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  737.532357] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 08f61a789eff444cbb3649ffa18c3935 in queue reply_156db6c4256a4c58884afce15e6f5340
[  737.541771] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 08f61a789eff444cbb3649ffa18c3935
[  737.659516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  737.660037] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  737.660037] env[60890]: ERROR oslo_messaging.rpc.server 
[  737.671807] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.672025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.672155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.672301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.672406] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.672572] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.672694] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.672756] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.672917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.673000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.673087] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 already deleted.
[  737.673171] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.968264] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.968477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.968655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.968874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.969025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.969296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.969454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.969584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.969936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.970055] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.970209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.970440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.970679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.970829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.970985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.971239] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.971395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.971517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.971727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.971901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  737.972166] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  737.972260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  737.972385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  737.972595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  737.972730] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  737.972954] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 47d03c13ccff4dd2a25ffd7fd6d2b97b in queue reply_156db6c4256a4c58884afce15e6f5340
[  737.977916] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 47d03c13ccff4dd2a25ffd7fd6d2b97b
[  738.016100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  738.016607] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  738.016607] env[60890]: ERROR oslo_messaging.rpc.server 
[  738.051230] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  738.051378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  738.051451] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  738.051670] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  738.051767] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  738.051865] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 already deleted.
[  738.052017] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  738.475379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  738.476038] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  738.476038] env[60890]: ERROR oslo_messaging.rpc.server 
[  738.554379] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  738.554760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  738.554760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  738.555090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  738.555195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  738.555457] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  738.555617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  738.555754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  738.555994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  738.556332] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  738.556462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  738.556581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  738.556789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  738.556933] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  738.557100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  738.557351] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  738.557536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  738.557664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  738.557884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  738.558021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  738.558259] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  738.558413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  738.558539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  738.558728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  738.558890] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  738.559132] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 78733c5553204fd7bd3c3966586c5b46 in queue reply_156db6c4256a4c58884afce15e6f5340
[  738.564708] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 78733c5553204fd7bd3c3966586c5b46
[  739.047751] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.047968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.048115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.048380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.048530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.048801] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.049002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.049092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.049313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.049560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.049713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.049831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.050041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.050186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.050335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.050582] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.050737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.050857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.051063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.051198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.051435] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.051587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.051704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.051944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.061723] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  739.061723] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ebd6423400594a4da2e5e89c1e17ec75 in queue reply_156db6c4256a4c58884afce15e6f5340
[  739.061723] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ebd6423400594a4da2e5e89c1e17ec75
[  739.061723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  739.062150] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  739.062150] env[60890]: ERROR oslo_messaging.rpc.server 
[  739.359039] env[60890]: INFO networking_nsxv3.common.synchronization [None req-5dc9ae61-458d-4eb3-96d4-70500e1f65a0 tempest-AttachVolumeTestJSON-389769224 tempest-AttachVolumeTestJSON-389769224-project-member] Enqueued Resource ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 with Priority: HIGHEST for action security_group_rules
[  739.359453] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 with Priority: HIGHEST for action security_group_rules
[  739.359769] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.359888] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.359952] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.360187] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.360267] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.360479] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.360588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.360647] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.360873] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.360959] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('2697824d-b6cb-4c8c-8cbe-cd77309bea77',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  739.361128] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3cb3ef0633dd4d50868e354b98fef5fe in queue reply_156db6c4256a4c58884afce15e6f5340
[  739.370618] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3cb3ef0633dd4d50868e354b98fef5fe
[  739.390998] env[60890]: INFO networking_nsxv3.common.synchronization [None req-df40137e-175e-46b1-853a-7d6f7a2904b2 tempest-AttachVolumeTestJSON-389769224 tempest-AttachVolumeTestJSON-389769224-project-member] Enqueued Resource ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 with Priority: HIGHEST for action security_group_rules
[  739.391579] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 with Priority: HIGHEST for action security_group_rules
[  739.392000] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.392436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.392627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.392936] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.393122] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.393385] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.393578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.556135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  739.556135] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  739.556135] env[60890]: ERROR oslo_messaging.rpc.server 
[  739.682742] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.682742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.682742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.682942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.683098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.683364] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.683513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.683626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.683842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.684098] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.684247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.684361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.684567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.684707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.684926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.685173] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.685316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.685485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.685726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.685912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.686155] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.686303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.686416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.686614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.686745] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  739.687003] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2f8e46dbcabc4faa83360d4e5a2ae128 in queue reply_156db6c4256a4c58884afce15e6f5340
[  739.693452] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2f8e46dbcabc4faa83360d4e5a2ae128
[  739.880328] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  739.880512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  739.880563] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.880771] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.880864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.880959] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 already deleted.
[  739.881040] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  739.881183] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  739.881439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  739.881439] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('2697824d-b6cb-4c8c-8cbe-cd77309bea77',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  739.881639] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5187a829c57e4ca98e82763dffe5c2b2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  739.891405] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5187a829c57e4ca98e82763dffe5c2b2
[  740.141827] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.142095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.142228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.142488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.142636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.142901] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.143060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.143177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.143407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.143651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.143803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.143922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.144150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.144297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.144445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.144694] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.144962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.145115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.145331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.145468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.145753] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.145928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.146044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.146251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.146384] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  740.146610] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 87b528b8960d4cb197e7986a41e2cd4a in queue reply_156db6c4256a4c58884afce15e6f5340
[  740.151437] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 87b528b8960d4cb197e7986a41e2cd4a
[  740.189055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  740.189550] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  740.189550] env[60890]: ERROR oslo_messaging.rpc.server 
[  740.401328] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.401486] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.401562] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.401837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.401923] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.402023] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 already deleted.
[  740.402108] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.649527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  740.650057] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  740.650057] env[60890]: ERROR oslo_messaging.rpc.server 
[  740.743013] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.743267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.743403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.743668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.743819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.744098] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.744254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.744966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.744966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.744966] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.745212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.745212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.745582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.745582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.745689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.745892] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.746058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.746176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.746384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.746592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  740.746756] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  740.746901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  740.747020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  740.747262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  740.747399] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  740.747633] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 76d8752afeb7411c9e7253867f71a635 in queue reply_156db6c4256a4c58884afce15e6f5340
[  740.764023] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 76d8752afeb7411c9e7253867f71a635
[  741.206581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.206780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.206911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.207179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.207328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.207585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-817 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.207741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.207861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.208400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.208654] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.208813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.208937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.209146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.209292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.209443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-817" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.209694] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.209846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.209964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.210166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.210300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.210533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-980aae30-b405-45cc-8c9e-3a70ba8a0f09 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.210682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.210798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.211001] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.211134] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('980aae30-b405-45cc-8c9e-3a70ba8a0f09',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  741.211407] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 67c473685eb0414381d13c11c685ee3c in queue reply_156db6c4256a4c58884afce15e6f5340
[  741.216494] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 67c473685eb0414381d13c11c685ee3c
[  741.249998] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  741.250471] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  741.250471] env[60890]: ERROR oslo_messaging.rpc.server 
[  741.713707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-980aae30-b405-45cc-8c9e-3a70ba8a0f09" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  741.714231] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  741.714231] env[60890]: ERROR oslo_messaging.rpc.server 
[  741.793858] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.794060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.794185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.794457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.794602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.794894] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.795054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.795202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.795424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.795667] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.795815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.795930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.796226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.796376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.796555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.796807] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.796999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.797065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.797268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.797397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  741.797630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  741.797791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  741.797987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  741.798151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  741.798286] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  741.798515] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bb614612acbf49928a72bf0758fc5dc5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  741.805647] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bb614612acbf49928a72bf0758fc5dc5
[  742.300737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  742.301270] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  742.301270] env[60890]: ERROR oslo_messaging.rpc.server 
[  742.845931] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  742.846243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  742.846282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  742.846544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  742.846697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  742.846960] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  742.847187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  742.847337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  742.847561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  742.847814] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  742.847972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  742.848376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  742.848644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  742.848837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  742.849030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  742.849321] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  742.849519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  742.849671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  742.849915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  742.850092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  742.850369] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  742.850565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  742.850717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  742.850961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  742.851139] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  742.851407] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1778fc1e1e544996a56e506f94979f5d in queue reply_156db6c4256a4c58884afce15e6f5340
[  742.857217] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1778fc1e1e544996a56e506f94979f5d
[  743.170881] env[60890]: INFO networking_nsxv3.common.synchronization [req-46b5533c-3eb5-4b30-9b6e-9a68a1ec4ad1 req-38f47b99-8aa7-473f-8ff2-97f1c2cb4f64 tempest-InstanceActionsTestJSON-1548267367 tempest-InstanceActionsTestJSON-1548267367-project-member] Enqueued Resource ID: c4b2c93a-cefc-4fba-89f9-5076b85229e7 with Priority: HIGHEST for action security_group_members
[  743.171420] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c4b2c93a-cefc-4fba-89f9-5076b85229e7 with Priority: HIGHEST for action security_group_members
[  743.171750] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.171878] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.171943] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.172165] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.172242] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.172401] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c4b2c93a-cefc-4fba-89f9-5076b85229e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.172495] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c4b2c93a-cefc-4fba-89f9-5076b85229e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.172555] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c4b2c93a-cefc-4fba-89f9-5076b85229e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.172811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c4b2c93a-cefc-4fba-89f9-5076b85229e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.172846] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.172921] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.172981] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.173173] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.173205] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.173270] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c4b2c93a-cefc-4fba-89f9-5076b85229e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  743.173452] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d947b3dc59544813b4b6d32ca55e56ed in queue reply_156db6c4256a4c58884afce15e6f5340
[  743.185985] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d947b3dc59544813b4b6d32ca55e56ed
[  743.262252] env[60890]: INFO networking_nsxv3.common.synchronization [req-7e62106b-acdb-40c5-aba3-2bbc6dc831dc req-10145653-7cbb-4c22-9c42-197e0db09c11 tempest-MultipleCreateTestJSON-1912904963 tempest-MultipleCreateTestJSON-1912904963-project-member] Enqueued Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  743.262252] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  743.262252] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.262252] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.262252] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.262473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.262473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.262601] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-08b77cbe-1697-46da-85d2-54fa8e8cc834 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.262704] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.262772] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.262916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.263056] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.263147] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.263205] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.263341] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.263417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.263507] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('08b77cbe-1697-46da-85d2-54fa8e8cc834',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  743.263683] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c170e12b98554288a74173613ef70050 in queue reply_156db6c4256a4c58884afce15e6f5340
[  743.274397] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c170e12b98554288a74173613ef70050
[  743.355067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  743.355643] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  743.355643] env[60890]: ERROR oslo_messaging.rpc.server 
[  743.672780] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.672994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.673148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.673438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.673596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.673859] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.674023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.674145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.674373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.674621] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.674821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.674948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.675160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.675310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.675458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.675708] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.675935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.676110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.676323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.676462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.676711] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.676859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.676976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.677266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.677406] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  743.677661] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2266a30efd384b63abdbec571ffcfbcb in queue reply_156db6c4256a4c58884afce15e6f5340
[  743.678717] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.678836] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.678903] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.679060] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.679776] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.680034] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.680121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.680187] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.680340] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.680420] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.680498] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c4b2c93a-cefc-4fba-89f9-5076b85229e7 already deleted.
[  743.680574] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c4b2c93a-cefc-4fba-89f9-5076b85229e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.685361] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2266a30efd384b63abdbec571ffcfbcb
[  743.769071] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.769071] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.769071] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 already deleted.
[  743.769071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.900139] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.900428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.900466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.900728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.900879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.901136] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.901283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.901397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.901602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.901847] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.901993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.902101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.902300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.902434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.902575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.902862] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.903019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.903120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.903321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.903449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  743.903680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  743.903836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  743.903950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  743.904172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  743.904300] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  743.904524] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6c388ea780274c43b13479d1c948d6d2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  743.910260] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6c388ea780274c43b13479d1c948d6d2
[  744.181710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  744.182250] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  744.182250] env[60890]: ERROR oslo_messaging.rpc.server 
[  744.406896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  744.407442] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  744.407442] env[60890]: ERROR oslo_messaging.rpc.server 
[  744.463322] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 677d5818dc284ea4b3eb258823c75775 in queue reply_156db6c4256a4c58884afce15e6f5340
[  744.474676] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 677d5818dc284ea4b3eb258823c75775
[  744.882800] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.882978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.883134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.883436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.883650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.884644] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.884644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.884644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.884644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.884916] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.885085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.885201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.885414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.885625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.885859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.886363] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.886607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.886812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.887092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.887306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.887644] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.887896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.888148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.888466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.888683] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  744.889068] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8770de149ba540668014b5a7394525fe in queue reply_156db6c4256a4c58884afce15e6f5340
[  744.903380] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8770de149ba540668014b5a7394525fe
[  744.977659] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.977864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.977987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.978256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.978395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.978649] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.978806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.978920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.979136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.979686] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.979968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.980168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.980374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.980516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.980662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.980911] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.981060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.981173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.981371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.981497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  744.981734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  744.981882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  744.981994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  744.982221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  744.982357] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  744.982585] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2037895eaf6445b19db83f6931a324c1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  744.988222] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2037895eaf6445b19db83f6931a324c1
[  745.397234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  745.397757] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  745.397757] env[60890]: ERROR oslo_messaging.rpc.server 
[  745.486812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  745.487359] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  745.487359] env[60890]: ERROR oslo_messaging.rpc.server 
[  746.043541] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.043854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.043892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.044181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.044340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.044598] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.044779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.044903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.045124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.045368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.045522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.045641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.045875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.046027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.046180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.046432] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.046587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.046706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.046915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.047055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.047295] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.047450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.047566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.047773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.047913] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  746.048162] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0e11aad355024e9f828114ceb406c967 in queue reply_156db6c4256a4c58884afce15e6f5340
[  746.053149] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0e11aad355024e9f828114ceb406c967
[  746.337334] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.337533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.337655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.337930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.338071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.338393] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.338474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.338577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.338784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.339019] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.339168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.339280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.339477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.339611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.339749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.340142] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.340314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.340432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.340633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.340816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.341001] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.341146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.341259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.341459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.341586] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  746.341811] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 12cf86df268d4617b4c90c00e7762673 in queue reply_156db6c4256a4c58884afce15e6f5340
[  746.347757] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 12cf86df268d4617b4c90c00e7762673
[  746.550075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  746.550586] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  746.550586] env[60890]: ERROR oslo_messaging.rpc.server 
[  746.843467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  746.844073] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  746.844073] env[60890]: ERROR oslo_messaging.rpc.server 
[  746.869797] env[60890]: INFO networking_nsxv3.common.synchronization [req-ba1aaa5b-b809-4e49-ae83-3a53bd72ad4d req-818ffbe6-f461-4803-9e1a-16affcbaf911 tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  746.870290] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  746.870587] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.870724] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.870793] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.871000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.871084] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.871241] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.871336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.871398] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.871605] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.871758] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  746.871852] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  746.871920] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  746.872082] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  746.872174] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  746.872258] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  746.872430] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d48d73c197704ea1996e508011f0ce47 in queue reply_156db6c4256a4c58884afce15e6f5340
[  746.880939] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d48d73c197704ea1996e508011f0ce47
[  747.184630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.184915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.185002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.185243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.185386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.185643] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.185829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.185953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.186168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.186469] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.186560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.186675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.186881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.187017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.187160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.187407] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.187565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.187680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.187882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.188020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.188255] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.188401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.188513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.188713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.188856] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f1cd2fe6-a982-4355-b8a3-8e7128f618c7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  747.189089] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 396b253dd2784ba5bcbe828c3a6eed0c in queue reply_156db6c4256a4c58884afce15e6f5340
[  747.198726] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 396b253dd2784ba5bcbe828c3a6eed0c
[  747.211775] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.211944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.212288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.212582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.212760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.213034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.213199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.213302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.213576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.213828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.213979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.214140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.214489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.214489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.214619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.214954] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.215113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.215240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.215469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.215722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.215918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.216123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.216248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.216537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.216685] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  747.216923] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 112b86fb172741e6aeb9703bbb5bff8b in queue reply_156db6c4256a4c58884afce15e6f5340
[  747.223564] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 112b86fb172741e6aeb9703bbb5bff8b
[  747.375792] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.376117] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.376234] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.376487] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.376606] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.376841] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.376975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.377112] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.377343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.377438] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.377538] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  747.377625] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.481940] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.482191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.482354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.482683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.482880] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.483173] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.483368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.483516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.483771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.484087] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.484289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.484434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.484683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.485063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.485126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.485416] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.486390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.486390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.486390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.486390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.486599] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  747.486751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  747.486869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  747.487176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  747.487266] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  747.487487] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 20411c4820864bb690c2b75dc6badcac in queue reply_156db6c4256a4c58884afce15e6f5340
[  747.499396] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 20411c4820864bb690c2b75dc6badcac
[  747.691335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f1cd2fe6-a982-4355-b8a3-8e7128f618c7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  747.691873] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  747.691873] env[60890]: ERROR oslo_messaging.rpc.server 
[  747.720109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  747.720704] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  747.720704] env[60890]: ERROR oslo_messaging.rpc.server 
[  747.990394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  747.990394] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  747.990394] env[60890]: ERROR oslo_messaging.rpc.server 
[  748.577909] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.578216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.578262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.578540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.578875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.578949] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.579049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.579178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.579392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.579605] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.579751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.579864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.580086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.580227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.580371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.580622] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.580776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.580892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.581102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.581238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.581475] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.581631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.581750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.582206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.582206] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  748.582427] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bd61cad61bf94b84bcc6bff8d1ccdddf in queue reply_156db6c4256a4c58884afce15e6f5340
[  748.584631] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.584840] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.584973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.585188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.585447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.585589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.585722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.585812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.586018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.587088] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.587088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.587088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.587088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.587088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.587088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.587368] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.587415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.587676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.587782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.587838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.588059] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.588208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.589483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.589722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.589864] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  748.590087] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e06ad3a7cc7a4e739c679fc07432d471 in queue reply_156db6c4256a4c58884afce15e6f5340
[  748.591107] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bd61cad61bf94b84bcc6bff8d1ccdddf
[  748.599711] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e06ad3a7cc7a4e739c679fc07432d471
[  748.737510] env[60890]: INFO networking_nsxv3.common.synchronization [req-7e62106b-acdb-40c5-aba3-2bbc6dc831dc req-b4974265-b01b-46b2-ad19-1fcff60c7687 tempest-MultipleCreateTestJSON-1912904963 tempest-MultipleCreateTestJSON-1912904963-project-member] Enqueued Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  748.738713] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  748.738713] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.738713] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.738713] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.738713] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.738942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.738942] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-08b77cbe-1697-46da-85d2-54fa8e8cc834 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.739088] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.739088] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.739213] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.739354] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  748.739454] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  748.739513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  748.739642] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  748.739721] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  748.739820] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('08b77cbe-1697-46da-85d2-54fa8e8cc834',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  748.740036] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b8ecf688ba1d438ab5c908f154fc4cfc in queue reply_156db6c4256a4c58884afce15e6f5340
[  748.748160] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b8ecf688ba1d438ab5c908f154fc4cfc
[  749.091756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  749.092307] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  749.092307] env[60890]: ERROR oslo_messaging.rpc.server 
[  749.097328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  749.097743] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  749.097743] env[60890]: ERROR oslo_messaging.rpc.server 
[  749.135387] env[60890]: INFO networking_nsxv3.common.synchronization [req-69824a1d-a508-477f-a96c-74820bef7be2 req-023c4b03-84aa-4053-b2b6-c6ae6fc8e6a4 tempest-InstanceActionsNegativeTestJSON-1017871433 tempest-InstanceActionsNegativeTestJSON-1017871433-project-member] Enqueued Resource ID: 834265b5-1ab3-419d-a793-ce5156217d0c with Priority: HIGHEST for action security_group_members
[  749.135919] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 834265b5-1ab3-419d-a793-ce5156217d0c with Priority: HIGHEST for action security_group_members
[  749.136281] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.136407] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.136474] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.136677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.136756] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.136906] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-834265b5-1ab3-419d-a793-ce5156217d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.137000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-834265b5-1ab3-419d-a793-ce5156217d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.137057] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-834265b5-1ab3-419d-a793-ce5156217d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.137265] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-834265b5-1ab3-419d-a793-ce5156217d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.137861] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.138092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.138092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.138191] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.138273] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.138359] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('834265b5-1ab3-419d-a793-ce5156217d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  749.138579] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a15eeb535e7449f58e1a4168d9a5b2d5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  749.146542] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a15eeb535e7449f58e1a4168d9a5b2d5
[  749.242128] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.242335] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.242392] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.242577] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.242673] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.242912] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.243039] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.243080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.243232] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.243308] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.243392] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 already deleted.
[  749.243469] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.474221] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.474431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.474562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.474867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.475026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.475311] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.475457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.475580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.475856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.476139] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.476300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.476420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.476635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.476780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.476965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.477255] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.477416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.477537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.477750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.477889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.478127] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.478281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.478483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.478645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.478783] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  749.479020] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a2ae93c3cbfb4b9f9cd8535dcdb27643 in queue reply_156db6c4256a4c58884afce15e6f5340
[  749.483940] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2ae93c3cbfb4b9f9cd8535dcdb27643
[  749.633196] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.633480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.633480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.633751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.633903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.634158] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.634315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.634434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.634670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.634999] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.635168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.635276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.635482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.635625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.635770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.636024] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.636180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.636370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.636501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.636636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.636873] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.637029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.637219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.637424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.637564] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  749.637800] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg eb068878d06344d294d2d8b19a49e0b0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  749.640512] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.640512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.640512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.640626] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.640708] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.640910] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  749.640976] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  749.641039] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  749.641171] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  749.641247] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.641333] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 834265b5-1ab3-419d-a793-ce5156217d0c already deleted.
[  749.641403] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-834265b5-1ab3-419d-a793-ce5156217d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.643009] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eb068878d06344d294d2d8b19a49e0b0
[  749.981092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  749.981635] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  749.981635] env[60890]: ERROR oslo_messaging.rpc.server 
[  750.030748] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.030980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.031111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.031383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.031539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.031875] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.032072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.032201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.032428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.032697] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.032857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.032979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.033223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.033396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.033535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.033787] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.033947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.034063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.034276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.034415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.034654] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.034844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.034967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.035177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.035317] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  750.035550] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f412891950f24b018a54ef44f3f22050 in queue reply_156db6c4256a4c58884afce15e6f5340
[  750.041262] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f412891950f24b018a54ef44f3f22050
[  750.139690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  750.140224] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  750.140224] env[60890]: ERROR oslo_messaging.rpc.server 
[  750.537991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  750.538793] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  750.538793] env[60890]: ERROR oslo_messaging.rpc.server 
[  750.576202] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.576458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.576699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.576896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.577055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.577330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.577468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.577589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.577803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.578051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.578206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.578324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.579060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.579397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.579647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.580056] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.580278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.580449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.580709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.581059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.581400] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.581616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.581794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.582056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.582254] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  750.582547] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 889148053c584e5da71a71077d3beec1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  750.588571] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 889148053c584e5da71a71077d3beec1
[  750.752027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.752027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.752027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.752027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.753181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.756027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.756027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.756027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.756027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  750.756027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  750.756027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  750.756027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  750.756027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  750.756027] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  750.756027] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8112f25b190140cd8c6feae875582bd6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  750.762896] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8112f25b190140cd8c6feae875582bd6
[  751.086683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  751.086683] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  751.086683] env[60890]: ERROR oslo_messaging.rpc.server 
[  751.211709] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.211919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.212067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.212342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.212484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.212831] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.212984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.213117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.213384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.213645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.213796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.213911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.214120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.214258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.214400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.214646] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.214964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.215084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.215292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.215421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.215656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.215884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.216054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.216273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.216409] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  751.216638] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 98658d3d0ab14651850c656af4a5ec8a in queue reply_156db6c4256a4c58884afce15e6f5340
[  751.224632] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 98658d3d0ab14651850c656af4a5ec8a
[  751.260370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  751.260849] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  751.260849] env[60890]: ERROR oslo_messaging.rpc.server 
[  751.680328] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.680542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.680667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.680943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.681163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.681401] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.681660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.681660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.681861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.682074] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.682302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.682302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.682493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.682649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.683678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.683678] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.683678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.683678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.683678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.683678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.683960] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.684074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.684198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.684402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.684588] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  751.684973] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 32305f9ac2ec482192038632eb9706c5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  751.708168] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 32305f9ac2ec482192038632eb9706c5
[  751.719089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  751.720767] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  751.720767] env[60890]: ERROR oslo_messaging.rpc.server 
[  751.754471] env[60890]: INFO networking_nsxv3.common.synchronization [req-3dcd480b-9a4a-4bdd-99c2-53118f539843 req-a95f0114-f2d0-4e0b-99b4-8b936f59c0ca tempest-ServerMetadataNegativeTestJSON-1300602226 tempest-ServerMetadataNegativeTestJSON-1300602226-project-member] Enqueued Resource ID: c301597d-df18-47d9-9af9-b449ba38748b with Priority: HIGHEST for action security_group_members
[  751.754471] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c301597d-df18-47d9-9af9-b449ba38748b with Priority: HIGHEST for action security_group_members
[  751.754831] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.755223] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.755223] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.755223] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.755364] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.756029] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c301597d-df18-47d9-9af9-b449ba38748b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.756029] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c301597d-df18-47d9-9af9-b449ba38748b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.756029] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c301597d-df18-47d9-9af9-b449ba38748b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.756276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c301597d-df18-47d9-9af9-b449ba38748b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.756431] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.756529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.756589] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.756741] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.756818] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.756902] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c301597d-df18-47d9-9af9-b449ba38748b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  751.757082] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ed30409eb5ff486e9b823fdf6464ad7f in queue reply_156db6c4256a4c58884afce15e6f5340
[  751.765267] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ed30409eb5ff486e9b823fdf6464ad7f
[  751.890981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.891190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.891319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.891592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.891742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.892015] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.892184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.892306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.892521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.892916] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.893073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.893189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.893404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.893549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.893731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.894331] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.894512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.894636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.895026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.895205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  751.895466] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  751.895631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  751.895822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  751.896100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  751.896248] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  751.896488] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 181778db034744b1915907e7c9b795eb in queue reply_156db6c4256a4c58884afce15e6f5340
[  751.908450] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 181778db034744b1915907e7c9b795eb
[  752.001392] env[60890]: INFO networking_nsxv3.common.synchronization [None req-aeb4b2bd-86f2-45b0-8b7c-99558e1f4035 None None] Enqueued Resource ID: 18da2e3a-855e-4c18-8f2f-947c2909e3c9 with Priority: HIGHEST for action port
[  752.001649] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 18da2e3a-855e-4c18-8f2f-947c2909e3c9 with Priority: HIGHEST for action port
[  752.002316] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.002397] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.002441] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.002607] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.002683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.002851] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-18da2e3a-855e-4c18-8f2f-947c2909e3c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.002941] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-18da2e3a-855e-4c18-8f2f-947c2909e3c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.002998] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-18da2e3a-855e-4c18-8f2f-947c2909e3c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.003195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-18da2e3a-855e-4c18-8f2f-947c2909e3c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.003267] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('18da2e3a-855e-4c18-8f2f-947c2909e3c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  752.003440] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e9dadcac155141fa8a4e8419c877bd4c in queue reply_156db6c4256a4c58884afce15e6f5340
[  752.034552] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e9dadcac155141fa8a4e8419c877bd4c
[  752.090603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.090889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.090992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.091225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.091363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.091609] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.091757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.091885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.092154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.092396] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.092585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.092701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.092854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.092994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.093136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.093388] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.093536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.093648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.093865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.093990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.094219] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.094365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.094523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.094709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.094859] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  752.095084] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg eba35fd0a9e44bd58a34bd4cc8ad23d5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  752.158407] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eba35fd0a9e44bd58a34bd4cc8ad23d5
[  752.188370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.188370] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.188370] env[60890]: ERROR oslo_messaging.rpc.server 
[  752.263718] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.264185] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.264352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.264649] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.264867] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.265177] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.265388] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.265531] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.265774] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.265949] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.266121] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c301597d-df18-47d9-9af9-b449ba38748b already deleted.
[  752.266285] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c301597d-df18-47d9-9af9-b449ba38748b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.407280] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.407280] env[60890]: ERROR oslo_messaging.rpc.server 
[  752.407280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.407280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.407280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.407280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.407280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.407280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.407280] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  752.407280] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 002f29e7fb234a6eb368868a1a07098e in queue reply_156db6c4256a4c58884afce15e6f5340
[  752.411287] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 002f29e7fb234a6eb368868a1a07098e
[  752.505799] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-18da2e3a-855e-4c18-8f2f-947c2909e3c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.505799] env[60890]: Traceback (most recent call last):
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  752.505799] env[60890]:     timer()
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  752.505799] env[60890]:     cb(*args, **kw)
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  752.505799] env[60890]:     result = function(*args, **kwargs)
[  752.505799] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  752.505799] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  752.505799] env[60890]:     return method(*args, **kwargs)
[  752.505799] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  752.505799] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  752.505799] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  752.505799] env[60890]:     result = self.transport._send(
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  752.505799] env[60890]:     return self._driver.send(target, ctxt, message,
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  752.505799] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  752.505799] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  752.505799] env[60890]:     raise result
[  752.505799] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.505799] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.597418] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.597932] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.597932] env[60890]: ERROR oslo_messaging.rpc.server 
[  752.756427] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.756844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.756844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.757127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.757390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.757671] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.757838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.757967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.758202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.758455] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.758614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.758738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.758954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.759101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.759251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.759506] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.759663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.759848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.760087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.760234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.760482] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.760637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.760757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.760974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.761113] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  752.763922] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ada901f61d4e4b5bbdd0acf407cdedef in queue reply_156db6c4256a4c58884afce15e6f5340
[  752.772920] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ada901f61d4e4b5bbdd0acf407cdedef
[  752.908433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.908911] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  752.908911] env[60890]: ERROR oslo_messaging.rpc.server 
[  752.945080] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.945305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.945437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.945739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.945929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.946179] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.946338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.946458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.946684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.946937] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.947118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.947240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.947463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.947610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.947761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.948040] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.948216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.948327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.948519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.948665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  752.948906] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  752.949063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  752.949181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  752.949385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  752.949522] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  752.949754] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e96025e7478c454cb5530e1edf989ce5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  752.961135] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e96025e7478c454cb5530e1edf989ce5
[  753.157375] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.157589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.157715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.158060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.158206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.158476] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.158632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.158798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.158958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.159200] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.159352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.159470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.159692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.159812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.159961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.160263] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.160412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.160532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.160739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.160874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.161112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.161265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.161381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.161589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.161723] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  753.161950] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 11d1101ca74540c79f9d50c62a121851 in queue reply_156db6c4256a4c58884afce15e6f5340
[  753.168942] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 11d1101ca74540c79f9d50c62a121851
[  753.267308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  753.267826] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  753.267826] env[60890]: ERROR oslo_messaging.rpc.server 
[  753.452646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  753.453151] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  753.453151] env[60890]: ERROR oslo_messaging.rpc.server 
[  753.454704] env[60890]: INFO networking_nsxv3.common.synchronization [None req-df1890da-0ddc-4bbb-9899-44c87777da0f tempest-ServerActionsTestJSON-983883571 tempest-ServerActionsTestJSON-983883571-project-member] Enqueued Resource ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 with Priority: HIGHEST for action security_group_rules
[  753.455358] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 with Priority: HIGHEST for action security_group_rules
[  753.455652] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.455942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.456038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.456238] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.456316] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.456467] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-68938528-e0ea-4599-af8e-22dc66ee1e55 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.456565] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.456628] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.456839] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.456926] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('68938528-e0ea-4599-af8e-22dc66ee1e55',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  753.457213] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c4cf57bfe9b740baab9d9a7c2f63c2e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  753.470576] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c4cf57bfe9b740baab9d9a7c2f63c2e0
[  753.531865] env[60890]: INFO networking_nsxv3.common.synchronization [None req-2e239019-c1b0-482a-b377-0ff19a762498 tempest-ServerActionsTestJSON-983883571 tempest-ServerActionsTestJSON-983883571-project-member] Enqueued Resource ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 with Priority: HIGHEST for action security_group_rules
[  753.532371] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 with Priority: HIGHEST for action security_group_rules
[  753.532758] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.532896] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.532962] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.533160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.533240] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.533391] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-68938528-e0ea-4599-af8e-22dc66ee1e55 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.533467] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.535199] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.535409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.535532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.535845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.535966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.536275] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.536428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.536541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.536754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.536996] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.537256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.537422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.537653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.537806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.538446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.538759] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.538947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.539071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.539314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.539455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.539702] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.539846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.539961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.540303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.540598] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  753.540973] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 15d90947484044cd866626dfaeec0900 in queue reply_156db6c4256a4c58884afce15e6f5340
[  753.548381] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 15d90947484044cd866626dfaeec0900
[  753.663996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  753.664580] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  753.664580] env[60890]: ERROR oslo_messaging.rpc.server 
[  753.978685] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.978685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.978685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.978685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.978685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.978685] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 already deleted.
[  753.978685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.978685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.978685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.978685] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('68938528-e0ea-4599-af8e-22dc66ee1e55',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  753.978685] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f1ecbec72cf64b39893a8aa7b54923e5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  753.986198] env[60890]: INFO networking_nsxv3.common.synchronization [None req-00baf1ee-4337-46a0-affc-3fed679349e9 None None] Enqueued Resource ID: 65fc9a8e-d462-4f00-ab65-de156c35c06d with Priority: HIGHEST for action port
[  753.986198] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 65fc9a8e-d462-4f00-ab65-de156c35c06d with Priority: HIGHEST for action port
[  753.986198] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.986198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.986198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.986198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.986198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  753.986198] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-65fc9a8e-d462-4f00-ab65-de156c35c06d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  753.986198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-65fc9a8e-d462-4f00-ab65-de156c35c06d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  753.986198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-65fc9a8e-d462-4f00-ab65-de156c35c06d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  753.986198] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-65fc9a8e-d462-4f00-ab65-de156c35c06d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  753.986198] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('65fc9a8e-d462-4f00-ab65-de156c35c06d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  753.986198] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f2ba3936f88a4ee6b2e228e7e175cd58 in queue reply_156db6c4256a4c58884afce15e6f5340
[  753.996505] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f2ba3936f88a4ee6b2e228e7e175cd58
[  754.002195] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f1ecbec72cf64b39893a8aa7b54923e5
[  754.045758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.045758] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.045758] env[60890]: ERROR oslo_messaging.rpc.server 
[  754.117682] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.117682] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.117682] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.117682] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.117682] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.117682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.117682] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  754.117682] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8532651b908c48fb961c56feab4d7c2c in queue reply_156db6c4256a4c58884afce15e6f5340
[  754.127904] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8532651b908c48fb961c56feab4d7c2c
[  754.184513] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.184513] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.184513] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.184513] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.184513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.184513] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.187184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.187184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.187184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.187184] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  754.187184] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0c3fa36301694438a9bd691f012bcf14 in queue reply_156db6c4256a4c58884afce15e6f5340
[  754.194093] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.194262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.195465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.195465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.195465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.195465] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.195465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.195685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.195770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.196054] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.196229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.196831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.196831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.196831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.196960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.197122] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.197302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.197426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.197646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.197787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.198235] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.198235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.198480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.198547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.198628] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  754.198855] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 77523e5438854797ad6f7fb5cbfceafc in queue reply_156db6c4256a4c58884afce15e6f5340
[  754.199942] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0c3fa36301694438a9bd691f012bcf14
[  754.208460] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 77523e5438854797ad6f7fb5cbfceafc
[  754.487756] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-65fc9a8e-d462-4f00-ab65-de156c35c06d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.488049] env[60890]: Traceback (most recent call last):
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  754.488101] env[60890]:     timer()
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  754.488101] env[60890]:     cb(*args, **kw)
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  754.488101] env[60890]:     result = function(*args, **kwargs)
[  754.488101] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  754.488101] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  754.488101] env[60890]:     return method(*args, **kwargs)
[  754.488101] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  754.488101] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  754.488101] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  754.488101] env[60890]:     result = self.transport._send(
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  754.488101] env[60890]:     return self._driver.send(target, ctxt, message,
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  754.488101] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  754.488101] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  754.488101] env[60890]:     raise result
[  754.488101] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.488101] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.497715] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.497839] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.497915] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.498131] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.498283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.498334] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 already deleted.
[  754.498400] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.620766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.621279] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.621279] env[60890]: ERROR oslo_messaging.rpc.server 
[  754.678447] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.678662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.678791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.679062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.679210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.679739] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.679949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.680108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.680344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.680606] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.680770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.680923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.681147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.681296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.681450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.681707] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.681863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.681982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.682191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.682327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.682567] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f7ceb644-b710-47d3-bf4d-0031a182d3ad taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  754.682745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  754.682871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  754.683085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  754.683222] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f7ceb644-b710-47d3-bf4d-0031a182d3ad',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  754.683458] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 47584e6e182540228f4c9013145ff6b4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  754.688453] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 47584e6e182540228f4c9013145ff6b4
[  754.700665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.701119] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.701119] env[60890]: ERROR oslo_messaging.rpc.server 
[  754.702468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.702903] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  754.702903] env[60890]: ERROR oslo_messaging.rpc.server 
[  755.159635] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.159635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.159635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.159635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.159635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.159635] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.159635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.159635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.160158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.160158] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.160234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.160351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.160557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.160720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.160883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.161134] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.161286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.161401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.161605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.161737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.161974] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.162124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.162715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.163064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.163219] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  755.163557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 66265e4d8d5d4a2aab196e36e794ac19 in queue reply_156db6c4256a4c58884afce15e6f5340
[  755.168484] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 66265e4d8d5d4a2aab196e36e794ac19
[  755.185585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f7ceb644-b710-47d3-bf4d-0031a182d3ad" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  755.186084] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  755.186084] env[60890]: ERROR oslo_messaging.rpc.server 
[  755.264607] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.264891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.265027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.265345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.265432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.265685] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.265859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.265992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.266214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.266463] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.266620] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.266740] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.267188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.267344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.267497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.267751] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.267909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.268048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.268265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.268402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.268640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.268794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.268915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.269122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.269259] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  755.269493] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 18ef7b6702854bc2806f5c01c877774c in queue reply_156db6c4256a4c58884afce15e6f5340
[  755.271325] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.271500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.271629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.271844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.271990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.272252] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.272429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.272521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.272791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.273054] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.273212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.273332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.274251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.274251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.274251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.274251] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.274251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.274572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.274572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.274683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.274925] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.275061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.275179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.275384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.275515] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  755.275764] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d45f8db9756741de8e21aefd0a80252e in queue reply_156db6c4256a4c58884afce15e6f5340
[  755.276637] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 18ef7b6702854bc2806f5c01c877774c
[  755.277271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  755.277691] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  755.277691] env[60890]: ERROR oslo_messaging.rpc.server 
[  755.280878] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d45f8db9756741de8e21aefd0a80252e
[  755.666074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  755.666572] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  755.666572] env[60890]: ERROR oslo_messaging.rpc.server 
[  755.783271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  755.783811] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  755.783811] env[60890]: ERROR oslo_messaging.rpc.server 
[  755.812678] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.812909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.813042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.813726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.813726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.813726] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.813886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.813973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.814181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.814570] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.814831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.814965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.815175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.815332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.815483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.815765] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.815918] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.816051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.816262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.816435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.816621] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.816770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.816891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.817110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.817254] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  755.817482] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 25b46f1bf25e44bc91f994497a7b7024 in queue reply_156db6c4256a4c58884afce15e6f5340
[  755.822700] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 25b46f1bf25e44bc91f994497a7b7024
[  755.977786] env[60890]: INFO networking_nsxv3.common.synchronization [req-7e62106b-acdb-40c5-aba3-2bbc6dc831dc req-abe0e91b-1d09-4ebd-93ba-cf08ae889bed tempest-MultipleCreateTestJSON-1912904963 tempest-MultipleCreateTestJSON-1912904963-project-member] Enqueued Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  755.978208] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  755.979380] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.979380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.979380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.979380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.979380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.979380] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-08b77cbe-1697-46da-85d2-54fa8e8cc834 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.979380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.979380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.979684] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.979684] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  755.979750] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  755.979782] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  755.979906] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  755.979977] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  755.980077] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('08b77cbe-1697-46da-85d2-54fa8e8cc834',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  755.980255] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5dc88ad7de054c658fcef0d70b7cddcd in queue reply_156db6c4256a4c58884afce15e6f5340
[  755.988183] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5dc88ad7de054c658fcef0d70b7cddcd
[  756.203718] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.203981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.204091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.204335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.204488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.204783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.204931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.205054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.205274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.205519] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.205672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.205789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.206024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.206169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.206317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.206562] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.206740] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.206860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.207063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.207195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.207427] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.207578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.207694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.207896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.208051] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  756.208290] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 47029cb8442a46c485d05951f7b214c9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  756.213290] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 47029cb8442a46c485d05951f7b214c9
[  756.319692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  756.320272] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  756.320272] env[60890]: ERROR oslo_messaging.rpc.server 
[  756.328209] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.328407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.328537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.328805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.328953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.329214] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.329367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.329484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.329699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.329944] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.330102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.330221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.330426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.330569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.330715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.331146] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.331146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.331262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.331506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.331646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.331887] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.332058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.332180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.332389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.332523] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  756.332756] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f656d9f5e5c34345ac9eddfbcb2ab420 in queue reply_156db6c4256a4c58884afce15e6f5340
[  756.338006] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f656d9f5e5c34345ac9eddfbcb2ab420
[  756.481667] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.482021] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.482310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.482688] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.482898] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.483190] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.483395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.483564] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.483814] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.483996] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.484179] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 already deleted.
[  756.484356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.711212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  756.711717] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  756.711717] env[60890]: ERROR oslo_messaging.rpc.server 
[  756.834913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  756.835474] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  756.835474] env[60890]: ERROR oslo_messaging.rpc.server 
[  756.869079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.869289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.869416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.869674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.869828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.870080] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.870237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.870359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.870574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.870817] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.870972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.871090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.871299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.871441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.871588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.871835] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.872053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.872173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.872386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.872522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  756.872768] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-32b7ce20-d3e7-414c-af46-db55c866bc41 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  756.872919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  756.873084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  756.873237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  756.873372] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32b7ce20-d3e7-414c-af46-db55c866bc41',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  756.873601] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 983ea0b0e0bb4672b7e0eebe07a70b48 in queue reply_156db6c4256a4c58884afce15e6f5340
[  756.879319] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 983ea0b0e0bb4672b7e0eebe07a70b48
[  757.258645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.258645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.258645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.258645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.259476] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.259476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.259476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.259784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.260041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.260329] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.260528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.260726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.260983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.261165] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  757.261493] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2df2f9f162b54d2f87bcc979b71214ba in queue reply_156db6c4256a4c58884afce15e6f5340
[  757.269835] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2df2f9f162b54d2f87bcc979b71214ba
[  757.377491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-32b7ce20-d3e7-414c-af46-db55c866bc41" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  757.377491] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  757.377491] env[60890]: ERROR oslo_messaging.rpc.server 
[  757.386532] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.386803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.386999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.387411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.387606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.387917] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.388108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.388246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.388530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.388889] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.389080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.389209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.389437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.389585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.389741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.390062] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.390275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.390406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.390625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.390766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.391072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  757.391239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  757.391364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  757.391655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  757.391810] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  757.392102] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e3f302837d1d4f248320c75f43c749f2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  757.398865] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e3f302837d1d4f248320c75f43c749f2
[  757.765194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  757.765194] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  757.765194] env[60890]: ERROR oslo_messaging.rpc.server 
[  757.894834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  757.895371] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  757.895371] env[60890]: ERROR oslo_messaging.rpc.server 
[  758.174841] env[60890]: INFO networking_nsxv3.common.synchronization [req-ba1aaa5b-b809-4e49-ae83-3a53bd72ad4d req-e33d258e-885f-4857-b8ae-d7b8165609c6 tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  758.175344] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  758.175708] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.175860] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.175937] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.176152] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.176233] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.176384] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.176475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.176532] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.176833] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.176833] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.176972] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.176972] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.177091] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.177160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.177246] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  758.177427] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0dcc3e7f8e5d4a63bd6aacfa7195291a in queue reply_156db6c4256a4c58884afce15e6f5340
[  758.190570] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0dcc3e7f8e5d4a63bd6aacfa7195291a
[  758.325244] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.325572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.325572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.325818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.325969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.326239] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.326396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.326511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.326733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.326982] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.327136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.327255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.327463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.327610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.327761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.328021] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.328181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.328302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.328513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.328650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.328889] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.329101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.329246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.329461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.329598] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  758.329830] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 20033d6ff7474bfcb51fda4d7e061585 in queue reply_156db6c4256a4c58884afce15e6f5340
[  758.334777] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 20033d6ff7474bfcb51fda4d7e061585
[  758.431447] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.431669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.431798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.432136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.432306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.432576] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.432765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.432891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.433123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.433370] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.433523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.433640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.433869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.433992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.434142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.434384] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.434536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.434653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.434901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.435038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.435279] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.435431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.435551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.435758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.435914] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  758.436168] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 904aa4b1f6954b4f874339d3bd01cb38 in queue reply_156db6c4256a4c58884afce15e6f5340
[  758.441704] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 904aa4b1f6954b4f874339d3bd01cb38
[  758.679381] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.679547] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.679612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.679837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.679946] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.680150] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.680251] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.680306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.680454] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.680529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.680614] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  758.680692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.818162] env[60890]: INFO networking_nsxv3.common.synchronization [req-6284ee71-a7ce-4c5f-98aa-6fc71a0faaf1 req-643cd131-3d99-4582-a69c-c089b3b235d3 tempest-ServersTestFqdnHostnames-535084876 tempest-ServersTestFqdnHostnames-535084876-project-member] Enqueued Resource ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b with Priority: HIGHEST for action security_group_members
[  758.818773] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b with Priority: HIGHEST for action security_group_members
[  758.819157] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.819310] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.819382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.819603] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.819706] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.819878] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-337aaaf2-15f5-46a8-8b59-236cfe63466b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.819990] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.820068] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.820299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.820448] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  758.820543] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  758.820654] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  758.820749] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  758.820830] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.820960] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('337aaaf2-15f5-46a8-8b59-236cfe63466b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  758.821173] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b061f9faca5c4bdab0c441f7402d3b40 in queue reply_156db6c4256a4c58884afce15e6f5340
[  758.830214] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b061f9faca5c4bdab0c441f7402d3b40
[  758.831874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  758.832349] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  758.832349] env[60890]: ERROR oslo_messaging.rpc.server 
[  758.939074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  758.939611] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  758.939611] env[60890]: ERROR oslo_messaging.rpc.server 
[  759.126188] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.126405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.126548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.126837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.127003] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.127288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.127460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.127591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.127901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.128156] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.128397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.128469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.128671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.128823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.128991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.129254] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.129420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.129538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.129760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.129908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.130163] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.130338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.130455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.130712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.130866] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  759.131111] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6f467990f41448d496f5135134a999a4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  759.136724] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6f467990f41448d496f5135134a999a4
[  759.323485] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.323636] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.323700] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.323968] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.324094] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.324292] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.324395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.324460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.324612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.324690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.324795] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b already deleted.
[  759.324882] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.369538] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.369862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.370032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.370323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.370595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.371011] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.371993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.371993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.371993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.372303] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.372366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.372476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.372687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.373010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.373221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.373491] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.373647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.373765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.374026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.374188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.374439] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.374594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.374710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.375099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.375309] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  759.375543] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b3b3304afe3c47d3bab023c34887e9ef in queue reply_156db6c4256a4c58884afce15e6f5340
[  759.381467] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b3b3304afe3c47d3bab023c34887e9ef
[  759.487374] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.487578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.487705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.487973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.488162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.488422] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-187 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.488579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.488699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.488917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.489161] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.489317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.489437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.489645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.489791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.489942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-187" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.490189] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.490345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.490463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.490668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.490841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.491118] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1089288e-9984-4055-87a3-b8addc9e8fe6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  759.491277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  759.491397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  759.491606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  759.491741] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1089288e-9984-4055-87a3-b8addc9e8fe6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  759.491972] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 64f8e71be3e9460d8c3061fd14383667 in queue reply_156db6c4256a4c58884afce15e6f5340
[  759.497701] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 64f8e71be3e9460d8c3061fd14383667
[  759.634913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  759.634913] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  759.634913] env[60890]: ERROR oslo_messaging.rpc.server 
[  759.877864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  759.878172] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  759.878172] env[60890]: ERROR oslo_messaging.rpc.server 
[  759.993625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1089288e-9984-4055-87a3-b8addc9e8fe6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  759.994153] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  759.994153] env[60890]: ERROR oslo_messaging.rpc.server 
[  760.187034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.187034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.187034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.187034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.187034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.187034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.187034] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  760.187034] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9c40fd7eefd54744a3b2c9d89e584826 in queue reply_156db6c4256a4c58884afce15e6f5340
[  760.192574] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9c40fd7eefd54744a3b2c9d89e584826
[  760.341782] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 97 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.341979] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.342061] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.342330] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.342446] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.342652] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock all taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 100 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.342774] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.342835] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.343028] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.343141] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_ports_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  760.343369] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0fc5528fb5ce47be99488de6f78d33f6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  760.350637] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0fc5528fb5ce47be99488de6f78d33f6
[  760.418777] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.419048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.419141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.419462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.419616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.419875] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.420055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.420179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.420390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.420633] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.420779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.420898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.421105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.421246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.421437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.421639] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.421794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.421914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.422117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.422252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.422490] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.422656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.422786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.422996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.423131] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  760.423360] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b165ecd0c0ed47e5b0a6a3904724c3f6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  760.430055] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b165ecd0c0ed47e5b0a6a3904724c3f6
[  760.692395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  760.692395] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  760.692395] env[60890]: ERROR oslo_messaging.rpc.server 
[  760.793439] env[60890]: INFO networking_nsxv3.common.synchronization [req-69824a1d-a508-477f-a96c-74820bef7be2 req-89b385c9-2d23-489e-a471-26a35d97993e tempest-InstanceActionsNegativeTestJSON-1017871433 tempest-InstanceActionsNegativeTestJSON-1017871433-project-member] Enqueued Resource ID: 834265b5-1ab3-419d-a793-ce5156217d0c with Priority: HIGHEST for action security_group_members
[  760.793439] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 834265b5-1ab3-419d-a793-ce5156217d0c with Priority: HIGHEST for action security_group_members
[  760.793439] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.793439] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-834265b5-1ab3-419d-a793-ce5156217d0c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-834265b5-1ab3-419d-a793-ce5156217d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-834265b5-1ab3-419d-a793-ce5156217d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-834265b5-1ab3-419d-a793-ce5156217d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.793439] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.793439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.793439] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('834265b5-1ab3-419d-a793-ce5156217d0c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  760.793439] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg caf841d59e474f39b95f55d48d1b4311 in queue reply_156db6c4256a4c58884afce15e6f5340
[  760.800337] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg caf841d59e474f39b95f55d48d1b4311
[  760.844923] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_groups_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  760.845106] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 72513d8b07ca47a7b2ef7b6c46a1d5ae in queue reply_156db6c4256a4c58884afce15e6f5340
[  760.852643] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 72513d8b07ca47a7b2ef7b6c46a1d5ae
[  760.854707] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.854977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.855107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.855413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.855565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.855972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.856228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.856363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.856610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.856884] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.857081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.857331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.857575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.857753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.857945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.858219] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.858384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.858506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.858720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.858871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.859142] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  760.859329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  760.859455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  760.859666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  760.859813] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  760.860059] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 92c738d5ea2c40cfb51cd9004a085626 in queue reply_156db6c4256a4c58884afce15e6f5340
[  760.864476] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 92c738d5ea2c40cfb51cd9004a085626
[  760.926369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  760.927342] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  760.927342] env[60890]: ERROR oslo_messaging.rpc.server 
[  761.290773] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.290942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.291015] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.291239] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.291327] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.291519] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.291616] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.291681] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.291829] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.291922] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.291991] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 834265b5-1ab3-419d-a793-ce5156217d0c already deleted.
[  761.292109] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-834265b5-1ab3-419d-a793-ce5156217d0c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.347186] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_qos_policies_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  761.348035] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg cd675ec1b15a4346a2c1be13167202a2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  761.354564] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cd675ec1b15a4346a2c1be13167202a2
[  761.362007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  761.362863] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  761.362863] env[60890]: ERROR oslo_messaging.rpc.server 
[  761.549448] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.549780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.549780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.550050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.550203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.550461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-353 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.550623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.550741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.550970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.551220] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.551379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.551501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.551717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.551865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.552049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-353" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.552306] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.552576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.552722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.552947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.553088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.553439] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-20628e1a-2b07-4773-bd50-e6014cfcd32a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.553842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.554019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.554240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.554384] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('20628e1a-2b07-4773-bd50-e6014cfcd32a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  761.554622] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f6df0af6f6064946821040edab4f1080 in queue reply_156db6c4256a4c58884afce15e6f5340
[  761.560481] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f6df0af6f6064946821040edab4f1080
[  761.849707] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Inventory metadata is going to be refreshed.
[  761.849816] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Segment.
[  761.904912] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.905126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.905255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.905527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.905681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.906142] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.906342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.906488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.906720] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.906986] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.907166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.907293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.907509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.907652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.907800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.908066] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.908219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.908334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.908544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.908676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  761.908980] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  761.909087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  761.909197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  761.909401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  761.909554] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  761.909769] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3e420c5d43c5488c95093739df2cf06c in queue reply_156db6c4256a4c58884afce15e6f5340
[  761.916361] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3e420c5d43c5488c95093739df2cf06c
[  762.057352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-20628e1a-2b07-4773-bd50-e6014cfcd32a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  762.057859] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  762.057859] env[60890]: ERROR oslo_messaging.rpc.server 
[  762.412129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  762.412670] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  762.412670] env[60890]: ERROR oslo_messaging.rpc.server 
[  762.550987] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 118 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  762.551289] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  762.551289] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  762.551497] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  762.564051] env[60890]: WARNING networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider [-] Duplicate resource with OS_ID: 992 ID: cl2-zone-992
[  762.564643] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  762.565978] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:SegmentPort.
[  762.596939] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock SegmentPort taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 120 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  762.597066] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  762.597108] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  762.597303] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  762.612639] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  762.613602] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:SegmentPort Is:96.
[  762.613735] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:SegmentPort Are:{'519bac90-d40c-494c-8eb9-6c53e239f10a', '32332759-ca87-4ef6-955a-325ad700200d', '03fdd0ff-044d-4309-905b-d51a3bab1b81', '55f117f0-ba76-491f-b5c5-7441d3ff7787', '9435dc4a-ead0-4afd-a099-668739ae29f4', '66d98031-6842-491c-9961-c1b4d0fd5d53', '0f875a82-d52b-4617-86ef-09c6a8aeccbb', '2126120d-06ba-46a5-a74c-77f3b18345ab', '17b157f7-bf15-4a5c-a271-9af72764322d', '474946e7-99a3-4290-959e-f2dc1645b999', '4fb6c40c-bf99-4b55-9f6b-ca451ded5287', '7e152039-3f45-45d1-b2a1-c7acb717f68b', '1ca72486-d800-43c5-907b-725bb958b3a3', '2a1bc88a-7641-4017-874c-f15ee8ec3621', 'd357f4a7-7fe1-4677-a32d-b2a6c5780592', '99c860bb-14da-4e95-b94e-d77d15224b7a', 'fb2c7956-257f-4d1d-8dbb-83b5350798c3', 'c4585efa-40da-4b2f-81a8-523efec14826', '7e3eabc0-1383-412e-a875-c4e73daf6808', 'f599de71-cb96-446c-b853-bfcdc671260e', '31125a4c-2800-482d-8420-53a798a29c02', '0cb4488c-6859-4b6c-b322-dc290dbf0f8f', '3e63d897-dbf9-4401-a6d7-1650e7c0ecdf', '62510d77-c23b-4044-a0f1-0f38a92caf2c', '11ab945e-c9e0-4540-8ddf-a57fc4983384', '0ecbd240-2457-42ca-b168-e67fa16f56b3', '156f2622-5bca-4370-8d7a-06130b978cf7', '6f69240a-5122-4e7e-87af-f288a47b5ea9', 'e758e5fd-8995-41a7-9fe7-7ce7894fb7aa', '1e891f4b-00a4-481c-a00c-0320f9e75e62', 'd43c8245-514d-4c94-80b7-c12dfe9a6434', '088ac320-7bf8-4ebf-9318-5c12a3de2e62', '78c543de-6fc1-464e-b0a5-b0db23f19be3', 'ea8f9597-af66-4849-94da-7357f4969707', '889eac8e-299a-4c2b-a8e4-819f8f54a261', '08b26599-acd3-4729-9ae4-7cef241ebca7', '046ad10e-2286-420f-b6f3-0d598b3fcc03', 'bc6fc679-542b-4d2b-838e-70db009ad31a', '64487042-dc1f-4bbb-8b1c-0ff06c270b8d', '45968fc0-6e74-43f7-862b-4043c3249370', '39825d23-3b58-4cdb-8fa7-5a9ff493c789', '2eea5bcc-a0a3-4963-a879-5bf0c1c38f60', '2763deba-ab9e-48b4-b5a8-f7082a993850', 'e9ad7926-1d42-4ff9-aab3-d520de5ae40e', '33359e95-dbc3-4199-9367-e6bd0129468e', '32f3a7a3-308f-4fc7-8898-ed5ad6161587', '88452c84-34f9-49bc-aa27-a6d503e47fa4', 'e1979b45-d8d8-4076-a153-fa64ba78b7c0', 'ace6f305-6029-421e-89d3-bcca9fd2264a', '06f70962-82ab-4995-b7ca-a341783a2c6b', 'ac9ba9d7-c9f6-482f-9cbe-4ac9659c3194', '2da5eafe-fcd1-44dd-b2aa-c1bc16ff3d23', '34dd8f0f-dd08-44fa-9865-37de198ce337', '02696c3f-3dd0-499c-9baa-8a8cf21253eb', 'a385dc0d-17c5-49e9-9685-8285fc885a67', '1b4cdb06-1445-4694-a8a7-25090c7c8982', '74ea2df8-d554-4541-bfb6-8a8fad4756ea', '28285b72-9ad5-4703-950f-d20766e1592d', 'c4c889cc-a303-49e8-b5da-c71c873abb02', '885d71ab-3cf8-44a1-98fe-fedf69f9f021', 'e798c106-4155-4e94-90dc-5591d8705c9e', 'f5808565-bb46-4620-9c27-c24fe7a3d1b6', 'a161c3dd-1d1b-49c2-816f-07c0cc3fe929', '6865e1e1-9000-4b73-b41e-4a4753755582', '271514e3-602e-4224-bfb9-99943b868420', '9674a2da-6ae2-430a-80a2-67bab8177b7e', '8e1ea150-2eec-4dac-80e6-7d1daeaf351d', 'dcde1b76-b76a-4b6b-bbc8-099834069781', '925b62f2-60ee-49b1-974b-66ad3d724498', 'e9d5bf46-3b2c-4088-a6bf-07b65f970ae4', 'f467c1f7-4700-4387-bb2a-249f697bbb6c', '7fcb3b5c-ae38-468f-aecc-8e3293f03e66', 'c4d845cb-64ec-4e2f-87a3-29dcdf4f4b2b', 'c8e81773-32ca-437e-9971-50cef2a264ea', 'd73acfa0-7f7a-4f3e-98ae-e9e2b940b462', 'a367d71c-4063-4a70-9c6c-4f60da50879e', '3903c4b9-6919-4b40-9189-942790370190', 'f1f40c6a-f2c1-4a74-94e2-cdf2e32e24cd', '6040c7d6-38fb-48bc-9161-2b810f0ffb1a', 'fc8312bb-e44e-4d35-b825-f0e74adf2260', 'fa51a02a-4b3e-479b-b703-759a3babfeb1', 'f1446fa4-56d0-44ca-8222-7ed499af9ee1', 'c9f3dd72-2950-4d13-9d1f-d1d7538d8150', 'e3e3d0bf-2787-48ab-9720-f7cb2a5d4ce2', 'f00d4feb-c74c-4e36-ab50-7989164b3eba', 'c5200c6e-cf74-4b1f-8377-44fbfd9e43ac', '48e0c1c5-8a6b-4724-a313-2b27b5d0073d', '194b348c-e895-4f8d-ab94-c33cb6abd8b7', 'c6537245-d078-41cd-b50c-1df8377c8fd6', '0cec1ac8-30f8-403a-a9b6-97dd88f439b7', 'ca735a43-ae10-4d73-851b-23c32b6e90a2', '0e5e87d8-236e-443c-b231-531694a623f3', '10d09f8d-b8e6-456d-85e8-1ecfa1dd206f', 'ed65677d-a1b5-4cb2-a8c2-fd130551ab54', '85bd5f38-c811-4e68-8e48-2bed178d2a9a', 'e937c7a1-6075-47df-be05-bce50cae3909'} {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  762.613898] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Rules).
[  763.117649] env[60890]: INFO networking_nsxv3.common.synchronization [req-3dcd480b-9a4a-4bdd-99c2-53118f539843 req-d94a209f-b9cc-4770-bd52-7e8235f7554e tempest-ServerMetadataNegativeTestJSON-1300602226 tempest-ServerMetadataNegativeTestJSON-1300602226-project-member] Enqueued Resource ID: c301597d-df18-47d9-9af9-b449ba38748b with Priority: HIGHEST for action security_group_members
[  763.118174] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c301597d-df18-47d9-9af9-b449ba38748b with Priority: HIGHEST for action security_group_members
[  763.118497] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.118619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.118685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.118892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.118995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.119142] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c301597d-df18-47d9-9af9-b449ba38748b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.119213] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c301597d-df18-47d9-9af9-b449ba38748b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.119292] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c301597d-df18-47d9-9af9-b449ba38748b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.119441] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c301597d-df18-47d9-9af9-b449ba38748b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.119550] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.119639] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.119697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.119900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.119985] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.120089] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c301597d-df18-47d9-9af9-b449ba38748b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  763.120850] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8769e50d3b81442ab7c6e77ab5a0567d in queue reply_156db6c4256a4c58884afce15e6f5340
[  763.128513] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8769e50d3b81442ab7c6e77ab5a0567d
[  763.222287] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.222287] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.222287] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.222287] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.222287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.222980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.222980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.222980] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.222980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.223123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.223298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.223435] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  763.223662] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8ed726095f58444ca3e6e285a9fab4a8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  763.226031] env[60890]: INFO networking_nsxv3.common.synchronization [req-e427e7c0-2dce-4861-a095-2ac7835ce821 req-efb3111e-9641-4688-8ac9-2fbdfa3c0c96 tempest-AttachVolumeShelveTestJSON-2132759461 tempest-AttachVolumeShelveTestJSON-2132759461-project-member] Enqueued Resource ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f with Priority: HIGHEST for action security_group_members
[  763.226800] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f with Priority: HIGHEST for action security_group_members
[  763.227081] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.227189] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.227436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.227436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.227504] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.227614] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.227679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.227733] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.227916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.228067] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.228155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.228210] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.228360] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.228470] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.228594] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  763.228839] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg eb14cd9cbb8a4c799010f1048a9bb710 in queue reply_156db6c4256a4c58884afce15e6f5340
[  763.247232] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8ed726095f58444ca3e6e285a9fab4a8
[  763.258469] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eb14cd9cbb8a4c799010f1048a9bb710
[  763.326935] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.327096] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.327165] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.327377] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.344938] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.346565] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Rules Remote IP Prefix).
[  763.500049] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.500268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.500397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.500682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.500860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.501134] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.501291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.501409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.501625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.502206] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.502403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.502532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.502770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.502924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.503076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.503324] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.503484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.503627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.503860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.504038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.504288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.504442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.504558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.504895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.505061] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  763.505326] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 93d62dc197094212a5081a0b412f9521 in queue reply_156db6c4256a4c58884afce15e6f5340
[  763.510932] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 93d62dc197094212a5081a0b412f9521
[  763.622956] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.623389] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.623389] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.623578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.623578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.623728] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.623816] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.623902] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.624099] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.624211] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.624305] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c301597d-df18-47d9-9af9-b449ba38748b already deleted.
[  763.624389] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c301597d-df18-47d9-9af9-b449ba38748b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.727641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  763.728188] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  763.728188] env[60890]: ERROR oslo_messaging.rpc.server 
[  763.730992] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.731892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.731892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.731892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.731892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.731892] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  763.731892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  763.731892] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  763.732164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  763.732164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  763.732164] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f already deleted.
[  763.732164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.007745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  764.008552] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  764.008552] env[60890]: ERROR oslo_messaging.rpc.server 
[  764.268859] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.269046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.269668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.269668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.269668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.269877] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.269981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.270108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.270313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.270556] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.270711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.270828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.271035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.271175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.271322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.272029] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.272128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.272223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.272419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.272537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.272786] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.273149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.273149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.273353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.273488] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  764.273740] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1df629209e974cc8b03ed2a60ed8aaa7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  764.283102] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1df629209e974cc8b03ed2a60ed8aaa7
[  764.557955] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.558168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.558293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.558600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.558749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.559058] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.559236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.559358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.559576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.559821] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.559977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.560115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.560325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.560483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.560655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.560905] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.561058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.561174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.561382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.561517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.561750] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  764.561929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  764.562089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  764.562397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  764.562537] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  764.562804] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e6a444c909ca4bf997e1671dbfc4ea74 in queue reply_156db6c4256a4c58884afce15e6f5340
[  764.569320] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e6a444c909ca4bf997e1671dbfc4ea74
[  764.775449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  764.775953] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  764.775953] env[60890]: ERROR oslo_messaging.rpc.server 
[  765.065171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  765.065690] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  765.065690] env[60890]: ERROR oslo_messaging.rpc.server 
[  765.674834] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.675054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.675184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.675460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.675607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.675952] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.676167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.676294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.676519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.676772] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.676929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.677049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.677255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.677398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.677548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.677799] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.677949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.678064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.678268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.678401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.678635] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.678785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.678904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.679197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.679350] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  765.679589] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6086640915704c458ecdcd91f5c8052a in queue reply_156db6c4256a4c58884afce15e6f5340
[  765.687079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.687257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.687382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.687614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.687757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.688057] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.688228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.688351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.688566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.688810] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.688968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.689089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.689297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.689439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.689589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.689838] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.689993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.690114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.690321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.690457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  765.690693] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  765.690846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  765.690969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  765.691323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  765.691467] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  765.691727] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 88a798ef665841fdb87b18dc5abf8fa7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  765.692777] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6086640915704c458ecdcd91f5c8052a
[  765.710199] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 88a798ef665841fdb87b18dc5abf8fa7
[  766.159517] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules Remote IP Prefix) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 970 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.159882] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.159882] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.159948] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.168698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.174048] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Address Group.
[  766.193860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  766.194335] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  766.194335] env[60890]: ERROR oslo_messaging.rpc.server 
[  766.196704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  766.196704] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  766.196704] env[60890]: ERROR oslo_messaging.rpc.server 
[  766.776645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.776797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.776923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.777203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.777355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.777603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.777928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.777928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.778167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.778298] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.778445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.778568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.778776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.778987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.779086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.779284] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.779429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.779545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.779751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.779886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.780141] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.780296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.780415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.780621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.780757] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  766.780995] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 41a5593651e144f6ba190b3ed6ee0342 in queue reply_156db6c4256a4c58884afce15e6f5340
[  766.783144] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.783317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.783443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.783659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.783799] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.784104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.784283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.784405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.784615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.784983] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.785187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.785187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.785457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.785510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.785631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.785882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.786032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.786146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.786348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.786497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  766.786709] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  766.786861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  766.786980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  766.787216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  766.787349] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  766.787575] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4678104d67b540f4a9525c5986da2c82 in queue reply_156db6c4256a4c58884afce15e6f5340
[  766.793858] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 41a5593651e144f6ba190b3ed6ee0342
[  766.804507] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4678104d67b540f4a9525c5986da2c82
[  767.290297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  767.290297] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  767.290297] env[60890]: ERROR oslo_messaging.rpc.server 
[  767.292917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  767.293333] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  767.293333] env[60890]: ERROR oslo_messaging.rpc.server 
[  767.859845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.860195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.860355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.860756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.860843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.861081] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.861240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.861357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.861575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.861813] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.861984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.862133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.862386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.862553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.862788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.863096] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.863265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.863401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.863632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.863787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.864061] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.864249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.864393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.864633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.864841] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  767.865122] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 41b0b24515844b64939e8cd608d37b3a in queue reply_156db6c4256a4c58884afce15e6f5340
[  767.870485] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.870690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.870847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.871102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.872139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.872139] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.872139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.872139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.872294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.872576] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.872831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.873040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.873271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.873419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.873571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.873824] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.873984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.874090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.874316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.874421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.874707] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  767.874842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  767.874966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  767.875197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  767.875821] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  767.875821] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3c25e02b4f8b407e890eec4f89442018 in queue reply_156db6c4256a4c58884afce15e6f5340
[  767.881032] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 41b0b24515844b64939e8cd608d37b3a
[  767.881503] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3c25e02b4f8b407e890eec4f89442018
[  767.882245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  767.882663] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  767.882663] env[60890]: ERROR oslo_messaging.rpc.server 
[  768.228377] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  768.228377] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  768.381635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  768.382158] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  768.382158] env[60890]: ERROR oslo_messaging.rpc.server 
[  768.412241] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.412496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.412635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.413021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.413174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.413435] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-280 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.413596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.413717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.413938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.414226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.414390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.414549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.414832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.414971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.415124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-280" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.415377] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.415534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.415684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.415918] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.416069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.416315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.416473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.416593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.416806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.416946] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b4c08f0a-d784-4a45-b0a1-b03197a3cb69',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  768.417208] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b48dcbe760314fc1bd69646da21d1ef4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  768.422599] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b48dcbe760314fc1bd69646da21d1ef4
[  768.908928] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.909161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.909295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.909601] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.909766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.910057] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.910232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.910354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.910610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.910885] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.911126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.911285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.911520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.911685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.911860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.912171] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.912356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.912485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.912787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.912974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.913260] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  768.913544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  768.913673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  768.913942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  768.914122] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  768.914774] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 722ab24bb7894db28c563b23237cd287 in queue reply_156db6c4256a4c58884afce15e6f5340
[  768.921585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b4c08f0a-d784-4a45-b0a1-b03197a3cb69" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  768.921585] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  768.921585] env[60890]: ERROR oslo_messaging.rpc.server 
[  768.921585] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 722ab24bb7894db28c563b23237cd287
[  769.117887] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Address Group taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 971 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.118128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.118230] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.118499] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.127571] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.132985] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Security Group (Rules) Is:1104.
[  769.133404] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Security Group (Rules) Are:{'6b38d532-e264-4d27-841f-0e430bb2d26a', '5a93c6de-4566-400e-b225-6fbc784def02', 'aa046b3f-6d44-4acb-8e58-50499e3f8d60', '839be8eb-d3ab-42d2-beae-bb4a6136fea9', '1efe66d8-51ee-4c77-bbc1-b0bff97bd6a0', '83621333-7805-450a-bf42-807dc0903fc0', '02fa8098-695f-4e4c-af31-9d2f20a9fc7c', '2c8665b5-7a16-454b-b09f-403fece4484c', '3c29ea98-dc38-490b-a8e3-abfdc118aa44', 'eb4ebc6c-f4d5-4c08-af40-def126bc447e', '38ad9e85-cb39-4189-9d61-20af96e8fe10', 'a84b8cf6-5370-462a-97e0-5c81e1e22633', 'aeb219c3-95b4-4de5-8fce-a41f8b49395d', '126abd34-fe92-4587-b136-35d78d6b2a21', '80e5b5da-5c33-415d-8683-524bd7b4691a', '2fdf09f9-efa8-4a14-a5c1-0e56ad631495', '01432ff8-955a-47c5-b665-9e3b030f909d', 'f8760856-2ea4-4f89-833e-7c672a2ef54f', 'f37d617f-64a1-4560-85e7-3656703fbd18', '214bfba7-572c-4d07-afe3-1421b94efaff', '26963d6f-35f6-46e6-8a33-94add9111f7b', 'ae07ce08-1e2b-447e-806a-b5f96013bf23', '001325e4-3b30-4168-927e-590103bce85a', 'e00ef03b-af7d-40d2-b5c4-f5bf948cc372', '9c9c140b-f706-4226-bbda-9950cb3e7e0e', '3b8642de-6fce-4721-aa49-9d1d63858bbd', 'f5fdf95e-aaff-43a4-8704-88046c16f0e2', '582e5156-771e-4f49-afec-56511f46df00', 'bedbbbab-f4e5-46f9-9732-8502ac4f5810', '94a8dda8-2e9c-4a69-97ab-ba0bfcc63981', '58c8aa49-9014-43f0-bf5b-d03cbade34c0', '577785e2-d134-40fe-b9a8-9eaf978fb67b', '0a023c7a-aab7-40a3-b867-63f307a7e3f5', '85d4d26a-9348-441b-a876-7f3546795321', '40faa2ad-a6a0-4e51-bc8c-a9fd92e740da', 'dcdd5ccd-96d7-43bb-94e7-a1099f4bfcd4', 'd7206195-0162-4ee9-a6b3-04ceb279bf3a', 'a880ae15-f8b4-4fdc-9f4c-4b0d5cb86453', '77128aab-c69a-4972-bea2-90cce0209793', '8b437b23-1e6f-49fc-a7b2-c532406ff181', '7662670f-bab5-4ef5-9237-479b1e61561d', '44d8f22d-5739-45af-a4ff-362aebd76359', 'd80d761a-2e72-4f5a-bb32-0e68160fbaf8', '3c210669-e296-4518-9563-be9d25f120da', 'ff43b22c-be14-4da5-b80d-36d8c54a860c', 'a30a9427-5e37-4471-9504-44c2d979c5b4', 'e72f919a-2482-420e-9b67-7355fec0316c', 'fcaf713b-1bcf-453c-8091-8e6f34635f0d', 'e014e8c4-efec-481b-9ed2-387c349047ee', 'a50769c7-88fa-4942-9e0e-4cb3595a5c30', 'a3c3f631-daba-43e1-8dec-871740520dc1', '78fd7d31-72e5-4a21-983e-ce48c7a7b713', '238101b5-c86b-4985-a776-3e973c0b60fa', '88520078-a760-425a-ab82-3525ef1c53eb', '92c9eef4-615d-48a3-84a3-cee217f08dce', '8837c3c4-5fbf-426c-a135-efc2f8c355a7', '37199e44-9bde-45c4-98fb-f8c75b0d3a5c', 'ba366f8f-c63c-4119-b420-2c303055cf86', 'f6c2252d-2268-4bb8-bc22-19eddc7610c7', 'aca54de8-c4f7-443e-aaca-7a111e51cc28', '2a81fc84-89bf-45e3-96d5-02a41988759f', '4a52564b-9d2d-469e-ab01-f09f3cdb4bec', '82de9bdb-0c15-45d4-9661-e40f8ec643f5', '13a8ff53-507a-470b-a2f2-fa671d340f4b', 'be36ad87-707f-4732-8a4b-162f66bfff59', '51a8e61b-e2a5-4064-9a56-68018ba339f2', '3b36b955-9c40-4a67-8695-6ebbe0cda689', '306a0b9c-142f-4a95-aa86-5296dbe02cc5', '8a7bf70f-b39d-4901-b42d-15428712deb8', 'a56f9f3d-b945-4159-bef4-429464666018', '6f226f67-b184-4a1c-9718-ac5c0b72e657', '1f67154b-3464-48f3-bfec-9852b358cbdf', 'feee80c5-5ba1-497f-bf8c-7b9bc27cfda0', '9e94d5b9-a9f9-438d-bf22-ab385101ba5b', 'bf9b416e-20fd-48ff-92a5-e238ddb61bd6', '10c57cd4-3944-4147-b429-b5b89c258f97', '670a4800-7ff6-4e55-9b4f-b46b5fb428a3', 'bbe619d9-4b8c-4abd-bc40-7b4017d42fee', '0d9bfa29-bfea-4478-8096-75c79e4dacd7', '0e948d3f-0f2d-4ff5-8578-e597302ee69f', 'e89dc789-cbe5-4c13-a583-6a1301d6acf7', '6e14057c-e840-4c19-9129-61edf2345a07', 'fb8ab3f7-dba9-492a-9091-63d448aff8f2', '0a4dd387-f79f-4f2d-a835-4a5e4fb9b5c2', '0dcb73a7-8912-48d0-a92c-49d62ca12e1d', '839c3392-f7df-4dac-99aa-faaa0ffbc1f2', 'd16d3b0e-751b-4387-9f22-804830b458a3', '1796c9a3-1cf8-4a66-9371-2d52508e5700', 'cbbc9657-8f2d-46bb-862c-6adbd8262def', '3bb43de2-ba3e-4750-9f47-debf358864be', 'ec9b93bf-6a59-4346-9df1-49b056a88ddc', '8b9ec5ff-f9d2-464f-aa26-034bfdedae22', '7011e504-95ac-4051-978d-74e49a6a417b', 'af98123a-50e1-4f30-848d-b652c92345a9', '7b3da12d-92d4-412c-a65e-9a5f565e9883', '14aedcd0-7413-4ff0-921d-0dead6637d4a', '68414d1d-514f-4108-8375-22274b3f2dca', '6cad175e-7e1b-470c-9598-263e629dba58', 'e266007b-24a8-4db4-a5fb-0b1cb7d27945', '18b4bffd-56b4-4aa9-b6a0-f69907fac144', 'ffbd48a8-331c-4efc-bcbc-83d55297b0c5', 'f2e485b8-428f-4e43-8b2c-236f117503d4', '2104385d-94a7-4ae7-8fb9-47feacce37d8', 'e1b59c88-2405-4b3a-9532-73440e2f5282', '37b44564-498b-442c-b259-5368844123d4', 'f2cf7c53-362f-4b7f-b49c-1c4e4bffc040', 'cc605c9b-1ace-499d-b8d8-fdb69ae3ceea', 'e497abb5-a45c-4bc8-b91b-ac72652a730b', '1f1db2e1-ec7e-4773-8ae6-022310685050', '2be9f8fb-fcb3-439a-802b-7c35d9fe987c', '4236e321-4341-469b-b076-101e05a1c634', '0fadcb76-b0b8-4881-b50d-7cc81fe937aa', '01b1ddc1-f902-4110-b9e7-1cb93ba198e3', 'b28c27bc-96c3-4d30-8b23-29cb54f22782', '6f0b1a90-314d-4230-9491-be3fd8160b48', '765a18e9-f395-4ddf-b33a-49588400d911', '0184a453-8434-420b-b0e7-af6dafd6be94', '6a14e5db-843c-4c73-a696-14fe0c37c9a6', '70195330-0f1b-46a5-b7d9-6b6bf9e4072b', '61737029-1c60-45dc-bf45-2b89048815d0', '8fb82bec-30c2-44df-bdf5-44d93a4ca5a5', '1550bb5e-b1a4-4e47-b433-714984a346f8', 'ea5b4a58-a322-479a-8f6f-40f420ac6961', '664e7649-5be7-4789-af43-679bf958c2c4', '0b100977-6974-4db9-ae62-ac6f9bde5f10', 'c6b86557-ee5f-4293-a23d-7962c990d0fb', '2804cc2c-4f0d-48d3-9112-48e6f6f774a4', 'ce0e684d-579e-4e3f-9712-705c0dcd4311', 'dc03a2e0-07d8-4b88-acd1-086899d59d81', 'b7705396-d1cc-4e43-b888-51eb0ddc7e58', '786ef165-7fde-4078-8740-fd9959125b95', '66387184-b7a8-4a37-be9b-a395c1435289', '4d89026d-4a87-43cd-89e4-854240fe7c6c', '41057d22-19a8-4ff7-ac1d-bc20e62be8cb', '4c8da2fb-5b24-4232-be92-e283831004b0', 'bb568b7c-9ed4-4a23-a24d-826aeae5bf41', 'e3654747-6ce5-4c34-9e69-227614e33008', 'a7c899e3-1e84-40d3-9d54-5c9ff71f8c66', '2e3533c7-7ed1-4a27-90f2-244125fcd07b', '2961c826-18bf-491d-91a2-2578410a18e5', '6f41e73c-277c-4be0-b93b-716a4236d8c3', 'e6fd6dd3-caf6-4e6f-8c22-0a6b73c984ff', 'ec10ba18-ca05-4a69-aac3-cb5e549941e4', 'a6f8758f-2516-448e-b4ef-6d330700c92b', 'fffa6845-9346-4dc4-8d2e-6da0881eac27', 'bb162c56-48da-4b1e-ac87-01508b852ed5', 'b6caa5d9-eccd-42f8-874a-8431edde4e92', 'f418017b-6e72-4702-8512-db7d24311693', 'd11614a3-711c-444d-9ddb-2c5adf8bdac3', '6f97d337-3e1c-475f-bd3b-7b17823178be', 'c22400fd-9ed7-4e1a-aa18-f1d92060ea9c', '0b55b0f0-df16-4b41-81d0-2ca9c5bebf24', 'a8261488-7d47-4511-87f1-4b70c625c70e', '55b71a8c-01f1-4e61-8b26-4ad3dae6dd79', 'fb4f905b-4a17-453c-b75e-b4ce53ef2d71', '4a1ff831-4b30-4354-bfb7-c0e0b77bf90c', 'e0d50e7c-efe8-4547-9e43-cb13e88bd4a6', '1947f963-faf1-407c-9995-b29c79427cb3', 'f380c991-18a3-45aa-9613-78b861f34dfe', 'f04d980b-f7d9-4cb3-9d9e-e4e7d419f9b4', '009cb6a7-916a-4da9-a2bf-c8d7a528d2f1', 'a6a45cee-bfda-4847-836d-5b6a15bb6d61', '1456471e-5a27-44c8-97d8-9cdb7d993ef6', '73ec4e34-3f1c-47b3-865e-c49be02a1450', '889f5e8e-b65a-4b4d-b007-72804b983a88', 'dccdd215-7b22-458d-9bc3-45907211a8d0', 'ca7c53f5-cf65-4e22-add6-b32f11f73781', 'd1234b73-ab53-4889-8eec-a220eb91044a', 'cc9e4024-2a52-40ee-b508-c42b42999bdd', 'a04c855f-2aa5-4c62-83d5-439effff24df', '04b45e82-4967-45f4-9139-e8c50f261183', 'd9c01ffd-4146-45cb-9ea3-3e3965976f33', '7be99172-ff42-4033-82c4-835b0a7cee87', 'cb1cefeb-b766-4610-8d3e-19e0c163381b', 'ab12b59c-62e4-4f5f-9fa4-4fab0bf511ee', '4ae77807-7394-4fa4-b332-6eea374322c3', 'a95dc000-d4c1-4bd7-bd1f-ff8845c97940', 'f0b34f41-c6bc-46e7-be6b-3d655bf3c2fa', '9f20533d-1fd8-4673-a050-284b825543ba', '59cd27d4-8ffc-42a1-bb7a-dc2c137b1933', 'ed9d63ef-ed9b-4000-9c9f-d97486bb6313', 'f198c82e-e1b7-4eca-aaf2-7c18868f4600', 'ce40d545-6550-4aeb-a7f8-a735b53b8141', 'af954bb5-a10f-4799-a3d0-10a0d55313bd', '577cd8aa-d141-4ac8-8b74-29ff70a034a0', 'b4e81915-13ab-4309-ada8-9791e009e78c', '9641406d-76e3-44cf-953d-d62bb3cc38ea', '3fad8bbb-41d0-4ba8-8a5b-2b0d23302f5a', '600ee48b-6826-48b5-aadf-b58abd34f5fb', '9291a66a-88d9-4f40-a93e-02b5b5d3a937', '464c0160-2f24-4146-b155-ef58aa98875d', '85306219-2678-44ae-8ee6-700b28ca0c16', '69dfed45-5682-4fa8-bf65-d94bf0a958de', 'eecea5f3-9ecf-44ba-a1fb-bb0e38a8bae5', 'b78ca953-47e4-4994-b97b-d5f30cb26160', '45da1fa0-516a-41f1-beaf-7b0047938b36', 'a0000fb0-481e-407e-9a95-a78fc3f44d25', 'd2c8d550-8a95-4b31-b00e-12fdb6ac3bc7', '4d6fe570-837e-4a57-8d3d-49be88b2101e', '3d623604-6f61-4d33-8501-7ad81f063617', 'c568d725-1788-4353-9dff-531c9bae7d67', '954fa2fd-1746-4b01-ad9e-aefde97c7f30', '01dbf841-f24b-4efb-ad32-65d63130f31b', 'fb5a9539-4230-48e4-9fce-b0cd6e6ab6d0', '63e33c7c-ad25-4466-9ea5-137a5a078ef4', 'fc596c57-8872-47e1-8e7f-e8f866d86176', 'be5b657d-31ee-4a84-9250-d007fe761bef', 'fbc889a9-3e3b-4f64-a82d-4a5e945ae91a', '1cc479c9-58ac-496b-b318-42eca2b4a176', '5b08f0e5-1871-47aa-b176-6c28e431cd64', '4e2e5f41-a617-4156-8637-a2b47a5a2bba', 'd1cacce0-33c1-460a-9c13-6c9bb82d5bb7', '1438c41c-3724-446e-9470-6b6d07775655', 'ee2b287e-1426-49c1-b950-eed64810790c', 'ee9399ac-b225-4d37-9bcb-ae77e14aa39d', '254d4d15-0a7b-4020-bc2f-72258c1c8eec', '1fdc786b-94fd-4481-9215-6709736862a1', 'af1f2fce-4e16-4cb6-8e10-37d3243d2e5c', '971a0755-80ec-4c36-bfd2-3d93101a5abd', '11dd23d4-bbf9-410f-bcbb-2b9b647cf24c', '3a0ee21b-ebf4-43dc-a5fb-920833d4cce1', '5c3fece4-9b5c-4a46-99e8-2a6e7a265c7a', '55860ec2-2462-400b-95bd-99a3222c063b', '1dacba0e-f104-44f4-9b0a-f9bd23c6b6a8', '58293dc1-5c7f-429b-ba74-e9d42d1666f3', 'a00cec11-d330-4a1e-ad66-91971ce5b9a6', 'e9c089e4-563a-427c-b932-4a216df08a57', '94a34b61-b616-4caa-9c26-c3155d20d919', 'd2afd0b0-c524-4918-969f-433a9ebacbd6', '143a19a4-7516-4d81-886e-da40672df71d', '785f1ec8-1ef3-403e-b97e-4a8a678a5939', '8afd7b42-456d-48e4-883e-5c3a5d379be2', '209e8f21-7a8a-4a2b-832f-6b32d1ec0ed9', 'd25ba1a7-6e37-46ed-86ad-4f4bfb7df314', '78b4e008-49cd-470c-ae8a-774ba969b7c3', '49e89fb7-3aba-428d-8f04-160ef9bcf002', 'a22e0506-a966-4a0b-a991-bb4a8408cff2', '30db5f24-5503-4749-b8e2-821dc53792e7', '6674dba3-bb9a-49ae-84c3-a2ddd00e8ad6', 'c0603c93-df1e-40bd-8ce9-6418a127d6b0', '5666f6ff-3d69-4235-994c-452c0fae18fd', 'c948f1fb-4350-4d5d-b1df-fb3a14ac4817', '3a3e9ade-6c09-4828-8407-26f3e688d5e5', 'c906dc90-d7ca-405d-a45b-815c94a71b7e', 'fecb9d13-77f6-4006-a1bc-bf7d840d9f5c', 'fb9757bf-6622-4822-8f32-038495aadc6c', '4896d82f-b449-4189-b263-fd11ab18490d', 'c84fcc4d-1a97-4cb2-8925-4874ee6137e5', '2cac0691-5a91-4653-8f6e-74a883d2af5d', '38111bd3-4034-41e6-9ccc-782c5ae900e7', '1ffbc453-234f-4598-abeb-445d743b2d73', '589f9f17-d562-405d-807a-d06ca5f19245', 'bc7a86f5-a363-4d65-a3d1-5cf5eeb5513c', '08bdd2f9-f2af-40e5-92c1-55503fd797de', '68ff0746-0354-4f85-bd0e-e18806b5264b', '09ebdcdf-8b9b-4517-a588-00e15d641284', 'b965b829-c919-45d7-8f57-bf91f7115ff4', 'e16906cf-be6a-4cd2-a595-03325aa3f6ac', 'd6b8dcbb-e14f-4f11-bb76-f75033b893f8', '9f12fd63-092b-4cf3-8856-28beeef3485e', 'db061e8f-981d-4327-af00-5df74d7c5d30', '5ffed754-cb53-48b6-b90e-7a84d20cbcff', 'e1836ab1-010a-4a7d-b4e4-17caa2e57c69', 'e9d3bb15-9510-4ecb-9944-7b670002f3db', '4aca9e0d-b421-4d5c-97b8-1ff1c05076ac', '8ac9c40e-78a9-4ada-ae66-4da570b5cd9e', '24153d44-a4e5-4967-970c-02cc1409cfc4', 'a2c2c5b1-61d6-480e-987c-36b99a885833', 'faa09b75-06b1-4c1e-923e-770d142e3387', 'fedc0335-bfdb-431c-9ebe-2d0c5dcca610', 'e817d23f-de35-451c-8fd2-ca9c50526fc0', 'a0e22a54-72bd-41e3-9f5a-2fb084d17ddb', '6a3f10a2-a6b1-4d76-934c-dd426e139ba0', '56dbb27f-1ce7-404f-8a52-a85fdcc16099', '42659b80-961d-4ac8-8dd1-c8abdd91e151', 'ffdaaf16-0c6b-4a84-8f94-ec37d83fbf93', '8a00232b-280a-49aa-96cc-4db769877bdd', '770d3355-bdad-40a7-9165-8c87037ccff9', '43da1381-ceee-4b61-9744-379c9888eebd', 'f73e1b31-a070-4800-9257-1b25323936c5', 'c44bf0b3-e403-433b-90da-1f246c6f80f2', 'b1aa9247-ff1b-4b54-8d12-128f806a3983', '69ff7038-e436-4385-9ace-76489f5274a4', '9c9b2b37-25f3-4ea4-9705-88a83120332f', 'fe26cac7-fdcc-4383-a30c-d8de8a4d3c21', '16bf5aeb-22b7-4088-8df3-2ec0f278a90c', '60b683d5-e141-4352-93cc-20bcdd311f74', 'c424aa8b-f99b-43d9-8517-385e52405670', '35a53a62-2092-428a-af42-c36aa4a2e4e1', '1d5b1c86-5db9-4624-8257-59f524e8ce7b', 'ccf33da8-9d54-45f0-b267-2705cf6a4186', '7dc11649-1c7d-4afc-ae36-afbc6a0eb062', 'e0fa8e7f-6095-4051-971f-3d56e8b47ee8', 'a43d51e2-fcce-4c08-944b-cf9ce5083ab7', 'e3450d94-5f06-487c-8f1c-328bc6fba19e', 'b1797cf8-cb1f-4ebb-ac16-6ed10106b9ff', '3af8c6cd-f2d4-4c35-8e15-38a6e8156123', 'dcf5b496-a31a-4753-968e-77837002ca21', 'fff09ab7-441d-4a0f-ae90-674a718ec4f3', 'a4f9530d-abf9-4fd5-b392-1c74596d99d7', 'fa8397df-4f59-44b5-9c06-5d5820e0bd7a', '964275dd-587b-4d13-a15c-5e2e765dc173', '8238829c-bcc9-47ae-8b4a-f56da8d9bc6b', '662725bd-a5e5-456f-b838-563b326e3a69', 'f7cb0d08-2713-4610-ba93-1d22dd5b1e0a', '8f3dd00b-3728-4286-be1f-f6049b5ede30', 'd36e049d-d9b8-4376-97f5-c845ff70dedd', '2c2b4845-5027-4121-b64e-4d71a1abc374', 'cd57e4da-789f-4a64-bbf7-ce1d90673fc9', '5b92549a-ed5c-49bd-b865-19eaa2482b09', 'b6bd256b-05fb-4b80-8c50-7ec02c917143', '346f57b0-657d-4301-97c8-44b7290e6430', 'faf7e41c-e9f5-404b-9221-0c6b012ee740', '3d87c3d1-4f69-4fcb-8b9d-da1f031cdf07', 'cec17d99-408e-4595-9216-0339c1b1a12f', 'c4bcdb1b-9690-4b93-a9b8-55e2e3a3ea9b', '8121b060-9737-4559-b17f-75f227ac9048', 'c4de4049-5c54-449f-94db-e624dc87ed63', '7a5788cc-73ae-44da-b85e-b25ef06234e3', '3a74b557-b525-4dc0-a0e7-e14d50c91d0a', 'cdb1148a-23a6-4b57-b907-03ffc91195a8', '823e793f-a8d3-40fe-be20-df3b5ad9179b', '5aa069f2-bf6c-4c7c-8fec-a00897397e56', '1cf9d498-da4a-4180-94d2-0d5c00afe650', '6aac90cc-9579-460d-b2ed-04a81c109de8', 'e544e2a3-735d-4503-afc2-47d009f7935f', '7b08d4dd-a0b4-43cc-9d2a-972b135c1430', 'c64b8f84-967c-4071-b291-34ba48437ccf', 'fbaf7512-2908-4177-8ed2-42a865d610d6', '62753820-42fb-40da-b36e-a401f377357a', '6a3758ec-79fc-48ca-aac0-e30d12d3b6d9', '3ca70d72-b4f9-41e4-b92c-8c029e1cd2d7', 'a65dbb38-6e75-41df-ba67-0e189e721981', 'ce9c7edd-74b1-4fc3-b808-b611a0c61df5', 'f7b396d3-bdfa-4346-b07f-e6b751236aad', 'f3778041-429e-4499-9e3a-0a0ec8f16936', '165d33c2-263b-401d-91df-0117a29be480', '68fd8c46-2475-4e55-a9b3-42ee20771cdc', '945fb4ab-2407-4ec6-95ab-d9be92246a1e', '6c4fda67-9ddc-4459-afa0-be683844dd0e', '25c8a813-432c-476f-aeaf-b73c7efc2337', '711a3536-173b-45d0-9b9a-4d98244ffc01', 'ce1fd1a6-558d-4261-bc5a-b9b1d3568adf', 'f8132790-8261-4615-82bc-c992a0dc4d0c', '92026d5c-2af4-4b91-9583-fc592f5f71c0', '1bf44a11-d803-436f-87d8-597ce52acd11', '388fd340-e59b-4d1c-9db4-11b4ca6a731b', '609deb9b-c3df-44cf-a011-5b0ae760bf55', '7f558621-b3d8-43f9-b03e-954ad5175cec', '9e35f44e-6f91-4f66-8b91-436d0a57466e', '97e79043-346f-442f-94a1-fe8e792245a1', 'a44385e7-cb47-4f8d-9530-d060cdbb30db', '3dd464bf-f81b-4f06-91f9-9cb3c015b4a3', '1284ecfe-4f71-4dff-a589-8ae7cec46a06', '4649c51c-cca0-4694-973a-c9be70500ecd', 'eac9fec0-2641-4426-8f8b-3a48a0b32821', 'ad07e91e-1c49-48ce-9ae1-335ee19c8982', '1446656e-6f64-47a3-8a54-95b85571d145', 'f1bf0c6b-53a3-4503-811c-e63935f6cedb', '623f3cf6-106a-4a41-b707-92839c6859e1', '17f1a49c-29d8-4874-b0ef-5dc9a69026b1', '73f26647-937f-4b0e-baf7-5f7d891ab022', '6566d6a4-8755-4145-9c53-c66b367852fd', 'acfd2e4a-e422-4863-b861-bd4c1cbfab87', '27eca136-22f1-4f3b-a5f4-47f2e3f5ef28', 'ba02ed24-3ea0-4d76-ad58-055b1ca60908', '1502208c-631a-4cc1-bd71-30d4da929264', 'd72bc675-a968-4981-ae8d-4b8104a8fc94', 'a41600f1-eaf3-467d-bda1-3490acc6c6ee', 'b6829360-ebaa-4356-b51b-2856c130a04b', '95e4fb59-1a70-4dc6-b2e1-3a1b5cd6f0cb', '5e85b3c2-afd2-486a-8324-3da82026e742', '584726a4-ee3c-4cfd-873b-5c09940aace4', '4da78a20-e58e-44ef-90b3-719d970e3608', '5452a771-253f-4a3e-a7d6-3878af982983', '8be25d2e-54f2-45ad-aec4-f0c41d0c6960', '8e6b865b-b83a-4ec0-9137-268a6bbf2d46', 'eed75b04-4032-4491-8700-12abb6192242', 'e18fb7e2-a107-49e1-8559-c8a453a0c12c', 'f1e12046-5ff0-48da-978b-5054e910d3d5', '4e7b5bb8-5dc9-44ba-bd86-01b2fcca4f35', 'e065b96f-c7a7-4109-9a98-0abc08279f41', '6fc1a1f7-48c0-466d-a902-be41bb2909dc', 'c3558061-f684-4797-b74c-55e8de4a383d', '279e5a6f-86ae-47dc-934a-ae06083151a2', '84b96d48-bc75-4aaf-8339-ae9a2bae6798', 'efc90b59-f1dd-4eea-99f9-3b9a58b88483', 'd140e637-4054-4d2f-9545-669a57df1ea8', '3db62012-d8bc-4017-ba31-98fb2b592936', '3e024ad4-7bc8-4446-a722-e846d3e48e0b', 'ede7815e-f2b4-4e92-a8d4-e7f530e83c70', 'f3ef8423-bbec-4546-8d54-c4d2d9eefb3d', 'c1939d81-71d8-402a-a72d-eed70e81ec98', '1af34663-8243-424b-b749-35aaff17c778', '9a21c4b4-f54d-4459-96c3-c9e6ae9e8e31', 'd421d774-ddc0-4979-a5a6-cef76679e898', 'a053d09b-356c-4582-8595-542852913eaf', 'bfc6b91c-e131-47a7-81fa-88e5eb3c5fff', '241e9854-bde0-4e7c-be1c-dea37a2f0e90', '4ca6f1e9-0c4e-474e-a1cb-d41125407f0d', 'd4253539-4006-4dc4-a613-30ebad87735b', '40da80a4-7fc6-4866-a0e7-cc1154336ea6', 'e2c554bb-3bf6-479a-983b-5e58478ab704', 'b8416495-ecf8-4e60-816a-0db10f2c5d2e', '1230ed50-7883-4176-9792-c60e73edc163', '2c269fc8-c778-4161-8be5-10533e66ac9e', 'c647a34a-44bf-41fe-a35f-1481ecc31b22', '0bcc9b1e-9da3-455a-a32e-4c1324c91bd7', '09e4d63a-c555-45b0-8879-05ed1216b496', '77b2202a-fd3e-412c-8aa7-ffc8b0cf8f50', 'fdae3cbf-c8b9-47ad-b362-9e1cdcc23b87', '4af1a18a-6733-468e-8cac-856f966abb42', 'c0996d40-8bae-4a49-965c-513c1adb118c', 'c20809a9-47ae-4379-ac6e-d799c88889da', '45a5677c-96a7-4a37-89d6-9b7035f0ab14', 'fa9bcefb-95e5-4595-89e6-77d7b9e49f82', '9ad76aeb-e965-4d2a-a9de-92f04fb9b8e4', '79f4e8e4-d10b-411c-b47f-7969eec84600', '7597ca1e-0b70-4a71-bb82-922c1a9b52a2', '608073a3-82f6-4577-b38b-71ede8712317', 'f42f230f-d6ce-464e-8025-66a23d253f1a', '02da5604-ce37-4d80-b7c0-dad352952562', '8829a113-df8c-4cc5-8530-5b9d1f318fd8', '6af90a39-eb33-40d8-a4a4-a656b94df580', 'c639d0f5-bba5-4c41-bbc6-c92ce0a01206', '332bb205-cae2-4814-85b5-1225b0d42faa', '757c7c0d-a608-4026-b7b2-8764639b83c7', '9bfa128c-a182-43b5-b5e4-ccccbe0be779', 'fe982f5e-8999-46cc-8ae2-5c0610479a91', 'c908b30c-f7bc-48e7-9b07-5f652939eb79', '289b8cb5-827b-4b96-b41c-146d1d612207', 'f00d8ffa-e5ec-4126-8232-b3e64acef8a1', '7a58b996-fabf-44f8-abb4-3ec0d3dbfeaf', '49e061d2-9378-4f1b-9816-71de3f03f01a', '29571673-755e-4d4c-9728-1abff3f8fd67', '6c0987df-8e1e-411c-b19d-adf095a1050f', 'adb1a1ad-e570-4fdf-9488-538bde8d32f5', 'ab612c9d-05e2-4ab1-a2d4-9146bd0bb6d4', 'c4a645d6-4eb7-4201-99ae-fe3faa482744', '1e051928-63f4-4795-9722-b10db743f215', '37dc3b34-d2d1-4a5a-8b3c-eb401e782bdb', 'bc493fef-5f41-4377-8a24-001b00f280e7', '0725ccf8-beed-4f7f-b741-fc54ff7e2ebd', 'f2f1f0e6-1821-4054-bdef-4f3c76348178', 'e911664c-510f-4c67-8ff6-d67968070e84', '218da72c-3ddd-423e-86f7-d0aed4bd1e47', '5e33dc80-b973-4d0d-ab5a-c6a45198254b', 'ca441aa2-6796-4db8-960b-a7977e434d43', '58a6fe6d-8bef-4419-95cd-9fc604bdc421', '8365d17b-d86d-43e0-852e-f3fda8eae2d8', 'e594aff5-132c-4d9a-952e-1fc74d243a12', '958c0b6a-6f8a-4f6c-8406-f556f75af78a', 'e40a313f-2d65-4606-90be-d168ff24a3f7', '4d21ad8f-4e3b-49a4-96fe-8f66364a0d0c', '1abbf587-b255-4721-b8f0-2b96dc32a646', '1185d64d-2d10-4e81-b958-ee93bab94ae5', '7cd32ef2-1ef1-4f42-84b4-1cdcb23ab30a', '56de7b18-ad03-4d6d-bfb4-aea2c90a6d47', '08f8156c-4637-4906-b6b2-fcaec641b75a', 'a3da1b8c-c49b-466e-b18d-613f8ddc3b05', '34a104f7-6762-4110-ae85-4f192a1db9a5', '92779f0d-dd69-4016-beee-33e8705804a9', '1f9b0305-e137-447b-b185-53a69184efd6', 'b49295ac-1d35-4294-bba8-6aef3b287be6', 'fba18575-c2d9-41e3-83ba-ec8fcf23c984', '79fe113b-9305-40b6-91b2-c9f6abd84439', 'd3ad96e0-1d90-49e8-af43-e3d38e6f7128', 'dbaec471-51bf-4ec1-9822-7a92e9f1b77e', '93dae16f-4011-4ff2-a6d1-c013cf6dbd77', '2c8fc7c8-df5a-4914-b0d7-83b71000b835', '5e83a431-918f-463a-966c-6b402e6021cb', '1be0f48f-0825-45a4-9b3a-913bf013e327', '6838c4a5-f996-4a37-b8a2-15e2a0a4ffcb', '2f8ba902-14bb-4b79-a17f-aeff5c95c84a', '8ecfd421-bd58-45b9-b323-979d6e3630b0', '580aa329-04e3-45e2-8acb-fe14cb26013f', '04e49400-b1fd-43a1-a29b-b2b7deffb80d', '131fa278-c4ed-4af0-b2f1-1fb805dc2a51', 'f8530d7d-3d78-4c4d-a62c-e627e9d28065', '87e1bac4-6dfd-4dd2-ba6c-bf8a90c5453a', '4a26f80f-e99b-40a7-af80-ddd13ab9d582', '6fffeb0e-5192-4855-b76c-325a13629478', 'b65e9824-88c2-4a62-aaef-e317a555dff3', '1244147d-e91f-4a43-9903-ec672032953a', 'a3e59377-ee7a-4002-a95b-02dee37cae6f', 'e261ceb3-7a43-41a8-b870-105307106d6c', 'dbea1d53-4616-40a5-b389-56118fc689c3', '0ecd2cdd-7394-4aae-935f-083440f44c73', 'f830a09d-6403-46af-b959-8c9068eccf11', '85e6e031-95fc-4ce2-9737-b9b457a11759', 'e42b080f-17cf-42de-9fde-5a75f5d6da7f', '28e519e8-50db-407f-afc0-b7bb0f37ecbb', '066a337d-30d8-4291-a38c-3c98191037df', 'f29fc06c-9bea-4105-8982-01646af59e1e', '28dd8e74-e4d3-49fe-9918-84cc4ec0f499', 'ee3929a9-b431-4bc9-9334-fd309a38f152', 'd664efb9-22b5-4dc3-b65c-0b8ac6b52c2e', '3398388f-e461-4c4f-8673-88cd40aee29e', '358a54e2-b53b-44b4-98a4-d301e8b0d87c', '583533dc-a2d6-44dc-a848-e70aad3c5ee8', '11623846-afc1-4b5d-a7a8-70cf65f99d71', 'b174a6c2-747b-41d1-b7fe-88d2ef3b94fb', '2f50427d-088e-47a2-a79f-c9f02938e184', 'f4b13771-8208-4490-9e07-10bb00f4ec2d', '74960b3c-125c-469d-b324-99edb8902762', 'a0ff327a-4777-4ea5-a473-953e99f48fce', '1d6fe346-cbba-4f0e-ad78-ce674e51562e', '21882b2c-8f85-4025-a05e-9d5ac5c7dfe5', '8676b965-8b56-49ab-b6a9-a54292a4c53f', '022cc406-b4f2-420a-b707-82b789638d65', '712394e2-8cde-4fc1-828f-1ea228fb85a6', '3f0f56ed-0942-4f28-b2ab-e015705a388d', '1ff1ff9e-a535-450b-b5e5-2b3aad2a92e1', 'ee0feec7-0fc2-4f57-8eea-f3d9e2999db0', '11782be5-5a4f-4d91-a1f7-66a85f90ce4f', 'd3681b6e-accf-4e7a-8599-9732dff3b868', '9809779f-51b0-45e9-b693-edca5cfa39d2', 'b9b299b3-6398-452e-9651-6189a49e5fea', '010843b8-b1a4-4ece-95e1-d8430a677f49', '7300c61a-eec4-49e5-a60b-1ac573d1c2ca', '3e8314ac-6de4-4fe0-b3b8-8090e7238d54', '53437029-9e9d-4417-8d26-d440ce43ab8c', 'd3a2479e-d6f6-4769-ab79-f8f399872ac2', '3dc7a690-4000-4337-9052-3c175d5b17d1', '984cf644-5297-4ea0-8e48-171a2b9c84c1', 'ed944c6d-d921-486d-96e8-2d616bba357c', 'fca489be-e88c-4540-b01f-3eaa4cdc376e', '777fabc7-034e-42c9-a21b-bd8346081baf', '2e646167-2df2-497e-a3a3-3c3ed4e65936', 'a68a8b48-b54e-4fda-b0fe-f939c87763ad', 'dfc6871c-16b2-426b-8d7e-f339a06594ac', '63047547-b550-47c9-b993-2e886acbf560', '5812319e-e240-4a68-ae1c-eff464ba0a7e', '7e185c0b-5d07-4465-b049-bd18fc2048c9', 'acc316ea-6132-40a7-bdaa-c82630a45736', '9faf5a46-083c-4779-a318-0a209d1bfff9', '1c25d563-9932-41d3-acba-b0ca5aefe8c0', '12afa242-3b02-471b-92ae-01a7a666d975', 'd27cb9f0-260e-457e-811f-ffd1448009cc', 'e638a2b9-395d-46aa-9f48-ad893802cb22', 'e16d1536-3624-47ea-a0b2-4be9049977d5', 'c9ef6859-d2a4-4586-8477-efafa7d2937f', '048381bd-f60c-40c8-85c9-867a04ffe5b5', 'd944bff4-eef4-466d-abc4-a0323e8486f7', '6baa86ac-ebc5-4808-94cb-30af7cca84ac', 'fa71937f-fc79-464b-ac33-3f84913bc5c1', '79cba46b-3a56-43d4-a711-ed1c27f8a8ac', '2ef579a7-8663-40f0-aef4-ba3069611485', '310f4a54-46a9-43e9-b8e2-df926bc1fc13', 'f61b7271-7b94-43b8-b404-a9037f54d120', 'ccd2e48d-222e-499a-8e5f-ab3d417b96c8', '5377676b-7e17-43de-9646-f00a238b562d', 'c824be34-b6bc-4635-aebb-4714244319bf', '3b813add-6d07-48bf-b3f3-300ea3fb0c85', 'd790a0e2-32f6-48b7-9c6e-dee360e54606', 'b6f88864-38f3-4ce4-843d-1807f88ec790', '56f5050a-0fee-4e03-a7fb-334fc44234f6', '186a9c80-281c-4fda-acd9-271fafd649ad', '8eab6bde-c670-45bd-b91e-094355c25bb7', 'c3abbce1-9d00-40a5-9d38-2aff1aed5129', '9bf93fdd-2fad-4c46-a7a3-896d816da7ac', '28d55ac5-c96d-4867-970a-5a69a0bfa928', 'e4d7cf20-4a9b-46e3-a36b-4dd1d1ce97da', '6dc0566e-ead8-43a5-be56-dec682c7c63f', 'f96e429c-b74c-411b-87f9-eba9d50e53d1', '82879c65-a05d-448b-b0f9-6a7083ed4590', 'cdd34ae5-533a-49a0-ae1b-2dcab7087683', '2effa546-1ce6-449c-b3d7-0564561ec4ac', '8fd62893-b488-41fd-96c5-61a7ff5af1bd', '440ee2f9-d226-4f4f-bde1-d14200833b29', '011229d4-5e90-407c-a670-49ffc98f79d1', '5cb60160-7c7d-4b59-9203-2c5ab07842f1', '65a29211-e00c-439d-a26b-bbb119eb9630', '4c8ba24f-6d56-4d7b-91d9-68aae2ae8f74', 'd3181ea2-2e2e-4151-82a0-6dd3bf9aabba', '1bd6158a-c5e7-47ce-b0f5-6ffdb35d17de', '8ad9e821-9357-4562-aaf1-7bfd3b93fb20', 'ead39029-9ec0-40ce-9bad-b260809b8ced', 'f77386bf-9902-4005-93b1-73c0f9f397bb', 'cde94ddc-3f20-4c89-8a3c-9bbb54d26a99', '78758cbd-9265-478d-92d9-b7cccd2f4751', '8687f805-b0a3-4260-8d2e-3c87b697ce0c', '99a5ca37-83e4-4e3a-9187-587f8a2caf20', '6f9ba684-3830-44cc-8ed9-29e1389d71be', '9dd5a52a-3035-4902-a140-1b44b469fb5c', '697e286d-b7ad-4447-9b92-81fcd59c3236', '60d17e33-b6ec-4501-8063-2b3951911a42', '8af0aa51-01ab-4f51-82c2-9d4649f04066', 'c2b5a36b-d306-4db1-a57a-bfc5066833ff', 'ace9ff04-b28a-44ee-9eca-c9822f577fcd', '047eea06-4eb4-41bf-8874-8bea4d8e056b', 'd7227627-de83-4167-b4e9-1360bb1c99a8', '761e0a99-e835-47b3-adde-5cfcc0d6e48b', 'fbcc04fa-12f2-4242-bdea-9c53ad092f13', 'ce4707a7-28e8-41c7-a022-0d4a7b2a02e8', '1f945598-5c9e-4cb4-802a-74d8682d8cd2', 'ee83cf79-3a92-4dc3-8a01-38c8d2817290', '17deaf2b-ca98-4873-bf41-35e0b79bcddd', '022dee22-57bc-48a6-aec2-307aee481dc8', 'a338029c-77c2-48cb-aaf9-d6708dfcc628', '3b9eb7b6-bc65-4476-bde0-75f07d1a479c', 'cb646236-e11c-4a30-a4a6-b40305062d2c', 'b7d15d9c-e206-4355-bc66-600e5727a2db', 'dc274eba-cdfa-493a-8b22-6e418157eca8', '04ce3382-304c-4c10-b79b-a288e40525d6', '9fcbfee9-0f98-447c-9669-0bc475f8554f', '9df3dcd2-7333-4940-abc6-2902bec425b6', '74965ae7-000a-4b65-8ae0-28821722a4c5', 'c0ea3df6-66d6-494c-af93-599cc8005459', 'a099661f-0f96-4b74-baed-ee0ab9870f16', '37f6587f-2a6a-4021-b5c9-e38c3842e331', '83ba575d-c051-48d8-ab84-0e7a4a0397a8', '3da14946-2583-495e-bc5a-eef658d15c95', '6a51f7e6-7131-46a5-8fd2-a4347e699df6', '4b3860e4-68b3-4a9d-bb0f-c79499310593', '5333c492-671a-4354-9d2f-7d73251f1c39', 'ea7a55e5-3749-40aa-84e0-9738398e69ae', '4adec779-4085-43f9-9749-a4f79f5519eb', 'bdf82a38-e0e8-4f61-a3a9-ab71dd33fec3', 'c2fabf99-bf05-4453-8f83-cc8eca721e65', 'c8670329-b9f2-4472-95b3-14552103e125', '4e7ba94f-e9c2-4941-b6ce-383d82c0f65b', '189500e5-532b-48d1-88cc-579b774e8f7f', '07029ca3-e0b5-4d9d-81c7-22db124dba19', '5dc731b6-a1b1-458f-be71-b41cca193acf', '7d9a6280-47ee-4957-8405-1dcc209613bb', 'd97a3c4c-df2d-4051-8c51-b3078efc48d5', '5715f1ac-2a29-4b6c-97d3-8fa5ae66c115', 'f7d16528-a3c8-4c12-a34b-1b98d3e2b218', '4d45ca7a-94ec-4d4e-8f32-20a567f835e1', 'a9d36583-5e55-4934-9f8e-d8d22497ad27', 'ca83cf8c-07f7-4246-9428-991ce4845a38', '2d918405-702e-48ab-bc40-3dc1da4a7a2e', '86a210bd-92f9-414a-a467-2a842cfadaa9', '80eff49b-b490-4df3-a771-4b0bdae7592b', 'cf4ed4b2-dbba-40f2-abe6-19802aadf541', 'eba3010a-a026-4e9e-94a8-ae0f77724cb1', 'a8068f5c-6f96-4bff-b79d-a3dcfeeaa280', '0abdc94f-1499-4384-b38e-c8823b487f21', '928bfa94-b605-4714-b124-5a25c5e42bd8', '7ed3d189-3a99-4816-a1c7-5678c119d8ba', 'cb30f59e-dd28-4a75-94a0-259942493875', 'dff956e7-f33b-4c03-ae57-9538dd622b91', '8bf15a9a-69cf-4b50-b60a-514b36ae96df', '8f174427-d4fb-4f4f-a59f-5e0345bdadd9', 'a17c90cf-7203-4d92-b33e-0f3831ffe006', 'fc182a83-5472-4e2a-a9dc-7e0012e483c5', 'c7afb8e0-c173-4c1f-ae90-a1c65093017b', '41613cc1-c095-4291-869a-df02557c715c', '7e7eb2c0-d1f3-4347-827b-0acfce995056', '84ab8378-05a9-4ffc-af1b-e77135596962', '733e3537-1785-4484-b0cc-33df13195ad2', '59f24c01-a820-46f9-9ec5-26ccc2bb9b99', '22a79425-10fb-432e-a949-01a37c22c495', 'd40ca857-6493-444f-92a6-52947467ad7c', '02d1f251-1551-4f4b-8ace-5a89a897fba3', '97aae706-e442-4648-a616-eb9daf306a4f', '9969d379-361d-4a47-a876-150d83e0b7ba', '802ebbfe-ccd9-458a-93a4-48509d998992', '5eeb1e61-0ea5-4f40-b98a-4169bde6fa30', '124df5fb-bccf-4daa-847f-7576261e09eb', 'c5e60296-9c4e-4eb9-9df2-26a6d27ac31b', '597170dc-c5e5-40e2-9686-a29391e2f08b', 'fe1c3779-aa50-4fd7-9794-23f1b10f447e', '3851d527-deb9-4e67-9ca9-46a76985faeb', '0902ee22-e88c-4f0e-85b9-5e83fe2f0b30', '4efb2bd4-7b01-43ab-8d5f-b80aac7a71a7', '19ae0fce-1d97-4bd9-95fa-aec4f8cebc52', '6d90d74e-28ed-4911-95e8-51ac37233027', '49e33507-c0b0-4bd7-8209-f6ce0dbf15d0', '0dbe9eea-5849-421d-a500-6cd545a2c12f', 'ec2ed287-28c1-48dd-893b-c8cda0a508b4', 'd51bc6f2-1e50-4673-8e60-eae257787176', 'b5cc1352-4268-4513-a03c-bf633544e2dd', '264dbc8a-7fa1-4220-8d5d-c008824f5870', 'abac75c2-3ed7-47af-8df4-2c1597b67964', '5fe05a90-e94c-4c88-a16f-d82799a9bef0', 'fb50169e-785d-4403-9853-adffc6b5171e', 'e70c7461-4be5-437d-b243-abf069ac3ae6', '6f561a81-b15f-43ae-9cc5-eeb0f9a38a71', '1503fb5a-3fe8-4bf3-9eb2-7a903286756d', '0f3d6f25-b150-4f89-8f19-47e05112e968', '7ee44e78-1c9c-45ca-9757-b59f1531492e', '6aeada42-ec65-4bdf-8831-113036a29ffa', '49cebfb6-383f-4c28-9b4e-9a8c875ca489', '3d817822-730f-4917-9401-22a2a406fe8e', '7d054de9-59a2-449a-83f0-aae1c12fd104', '2b1d43d1-9bc8-4fe7-9f1a-04f6c98097ff', 'a44e1a27-2f77-4160-825f-63e89fc2e3a4', '29d3dc08-b4fd-4b56-815b-5f9fa7d351e4', '7fcfac3d-00eb-4bb6-a342-5833b1f3037c', '02a71b1b-d156-46fd-b8e0-919d3e68ab67', 'e2d02080-838f-441f-ae03-5b4e77cde9c1', 'f7c506e4-0ce6-4792-89cc-3ffd47246bec', '69e19bec-8ad6-406b-80cf-51ab5d80b65f', '37ba87ed-b988-42ab-b71f-19a26582f05f', 'eca7eac3-64d5-4db8-aac1-7d2171b5b594', 'e010414f-d677-4efe-99da-b7b66755d4ca', '6bfb0168-5032-40ec-a757-5f9ceeee0c65', 'e2d76a70-0ae1-40b6-a610-899a66e92716', 'ec03565e-b463-4894-a441-f6eaa1e14497', '1fe06423-042e-413e-aefb-31ea7173c770', '7c4d266b-9ceb-4335-943f-3decf874437f', '09c24c3a-991e-4e6c-86dd-00775ddc9db8', '1a1f374c-636c-4ffd-bba9-795be4d7c2f2', '663a0264-041d-4bed-b616-6e20aeb74b39', 'f43951b3-e42d-409a-aa7e-19dff1102007', '9d22157f-08d5-4c08-9fab-28d2e2ae7b57', 'c6ee3756-e8bb-44c9-b0bb-119e9f03b360', '34bf7959-f4bb-401d-b934-7920d274a088', 'a7616d94-b3fc-42e2-98b2-03bd04c97d0f', 'd312af10-76f9-43c3-9118-db3f81bde31d', '48c74bc3-3e15-4ba7-a5ac-786074cab253', '3e87029f-6dd3-4cbf-8d6e-a4f83bc8bc19', '9c49fa14-3e11-4b12-af33-d1398946ffc3', '9985ed29-19b2-4ca2-a114-d4a5db55a349', 'cf2c4e64-dad1-46be-befd-f8dcc859abcd', '8d96ae9f-9dff-4863-abf3-283f80f1027e', 'ee3322a3-875e-4848-9aa9-bfe069f0ae65', '9e62b556-144d-4381-b9f4-c58077f08685', 'e1b38e4c-4612-4d4e-9489-9953af45bea0', 'e191de29-8260-484b-a6a8-4d5ac2d0faca', 'f41d25da-3599-422e-bc46-f1b0d4c38eba', 'bbf86abc-e599-4bae-b673-973dd4c77365', 'e782a415-6fb6-40c7-b28d-d91e58ed44c5', 'b20561db-0067-42ad-8721-9326a247ffcb', '4481d451-0060-4cd0-87ed-b9a00dee1243', '7f811a3f-c3c9-4981-aecb-6bee8144bc66', '141d3859-76fd-4cb7-b64a-8e89bb6105e2', 'ad19bfc9-a815-4cce-a18f-1e50273a4940', '29ba3db3-4c1c-48c2-9785-aa587880a1d7', 'd534f8cf-207f-4b9c-b3e7-63e5396c976b', '91066cd2-7197-4c51-9c8e-3ac6a7f8685f', '55ae8638-cc22-424d-bc8e-304f641175e1', '21ddf401-722a-4898-b34f-e3dbc81ef122', 'aaf28ac3-74a6-4b67-9d27-2fb8b180f9e3', '5768f84d-bc45-479d-825a-5974920907ee', '1d3ba8be-748f-4508-83ad-056e89ecd39f', '70e9491a-e894-439c-b190-469564028fee', 'fe5e2980-4176-4c9a-8e8b-9cd47c9f7e07', '2d528d3a-e365-4f26-8393-65491081b279', 'bfaa69dd-d009-4377-be48-167890cd0179', '1ecf7494-7210-436d-b926-9e873e5b6529', '0ae7bd49-a7c4-4a09-8011-f70d483760f4', '95cf7921-2ddd-4b0a-b393-82942167c148', '8220e6e0-1274-4067-9362-d1843efa59d4', '2664d12b-7727-4ba7-bb20-6d36d52f12f0', '70edd117-06a0-4f49-bf35-29f4c7de3064', 'ccf74d3d-94c4-4ec0-8f39-eed826c566f5', 'ef765d30-9529-409a-927d-f25515bf1710', 'b3b126c8-2a60-4802-bbaf-1b098e51ecce', 'd015e83c-3cad-4f5e-bd30-4dbe071befaf', '8986e901-9e9b-4ed2-b31d-f2176a202a62', 'e0e50c73-fc45-4794-99e6-aa137290a4e1', '7572b39c-6969-487d-be36-ca0a69fdcb8e', 'af434786-c3ee-46f6-8d10-acc5ec0bec29', 'ba575443-b4fb-48f1-a008-c3aea0628271', '5dfa1861-20e7-4866-b924-cdf621dd0a88', '548d42ba-3381-4893-a39f-2237d2752a23', '9c6e8871-295d-461f-911e-2a16aa881389', 'b4b49aea-8d72-4ee3-81d5-398dd7085be2', 'eb27e4a2-d332-4a29-abcc-6ca01d089d06', 'f4f4208f-01e3-4f3a-9135-4574a328dcee', 'fd1cddde-992f-488c-b63e-eafcae157326', '9e561b7e-ec42-4d1d-9695-d93010948fa2', 'b5b4a96c-c1b6-497e-8191-a4b3f0bc39be', '51b22c8f-efd3-4377-8141-3b16d0476e5d', 'b25ae027-ca8b-4a81-bf78-94ec4c0d66ab', '34b97a40-57f8-4326-bcea-73dd6f2309a0', '7a3f1db2-3f97-4d58-b359-59fb7cd9ca49', '48df3a64-b065-4d84-ad45-92911f48d13e', '8b648bb5-7a71-47ce-811c-2049fdc0273e', '0676c759-6a02-4f46-916f-8bd3c99563c3', 'fe74f926-5f68-454e-a893-149be82f3346', '457f3c6f-f4ec-4749-97f9-117dfc06d4db', '5a728d3c-d769-4afd-853d-9d2552d239a5', '86a1e722-f662-43fc-bd17-037bd6123f06', 'ce3871ad-3efb-4785-a306-e49923889613', 'd1f00b2a-1df6-4873-b29e-3aa53818d384', '1322bff5-1e46-471b-a79b-e5da27677247', '1185d178-8e44-47bd-b4dd-a296e2bf9031', '3d4e3921-e785-454f-8ff1-f625205a8808', '23896746-3201-469b-bdd7-a0c2d3f396c7', 'c2e5ea10-1af1-4814-9cd8-ec569c0fb0de', '447eb45f-4d31-4eb9-98a3-cb5c05fbe1e8', '2e4464aa-97ee-4116-8c9d-b9c99123e0a1', 'e874d62e-9f2c-47d0-b5bf-84a9765b48e5', '1bebebdf-5387-4a03-b616-d4766e9e0630', '1b49a31b-1828-4674-9db4-1823b2c9d607', '31dd60d6-8b0e-4abc-81a3-e7bf5aee1cb6', 'f94f9ada-e246-472f-8fea-233c1b8bbd47', '24a7e24e-8902-4872-acc4-aa3665594333', 'fb4837ae-9d8f-4e78-8039-45d62225e224', 'e0add8c2-9f3d-4557-8317-3fd19c913636', 'd1459ed9-6cbf-4387-ad59-972de8282692', '5c026cf8-d0c0-4784-8058-3a035144b3da', '2d157506-234d-4988-8bd7-88e7c35d19b0', '47420f4c-9e29-401d-a29a-1341d1127262', 'd43ee8ea-6cca-4f7f-9bce-a3a8e5e7dab4', '2645464c-d1cb-4d10-9075-17315791f00a', 'bb97eb22-25db-4f75-88a5-abf0eb3c8af6', '35bf7f9c-a62d-41f5-a3ba-99402d1f1eab', 'c4f807a4-41b6-480e-b973-86ff7e73ab21', '22138115-c9b8-4e58-aa35-367f13b91e17', '92de6bd1-222a-4fb5-bb41-120db189a0f7', 'd46b57f1-3a75-4338-8808-65582f8d94c8', '9f584829-b266-48fd-98cf-e86a4235fcbe', '17aab31e-3103-49c0-b8d6-9759bbe79a84', 'c14c1e16-eae8-4d5b-9ddf-b72d64c43125', '1492808e-c3c2-45e4-8518-cab2612e22e7', '66caa0a9-74a8-4cce-89cc-2312d253c6f7', '5a8460ef-028b-44c6-a4bd-d59dcb0796f7', 'ed20d25b-c10d-4f19-a527-ba3986779d48', '415de96b-e6bb-494e-b35e-966c7dd7a87f', '84fe4c94-9dd2-4322-9fbf-1016ee227291', '27fe07e1-d4f3-44bc-a492-2f9d1ddeb0c7', '302ea960-1230-4bf8-99fe-27b1f9067cc2', '894364b3-fb27-4540-a326-1f5d61b78745', '1684acf9-ef02-4e79-a090-bd3e2a427f05', '023631b0-dcd4-45ee-b15c-814f0cf38a3b', 'd6093c90-218a-4dfa-b851-e6683be1b50f', 'c7a0ff84-0f87-41e5-a8da-5257d8f22fbc', 'b6b65099-73c3-4abb-bd1c-842f26e18531', 'ba4e3148-fcf9-42fb-a0b8-8ab4b52ee982', '1a5ed077-8c88-47e4-ae7e-2008eb4c3e22', 'b6071397-8be8-4827-885a-45bbc6d1fb4c', '9feb48b1-9a04-4ba0-bc24-f63ecedfe23c', '3c6cb31b-9f2b-49df-8b06-50468e85fbf8', '31622995-c2ae-494f-b685-3bc348ebd21d', 'c08ae808-8493-4daa-ab35-daa380ee9df8', '80f8c2fa-c4c9-431c-9ae8-23007b27a85c', '34a888c2-3361-4ab5-a980-e3bfa6efc30d', '0904f757-7efb-44d1-a07c-24ff0833a5bf', '8c4347f0-f777-4bda-921c-83dedf80e1de', 'c92ee54a-0b8e-4a1d-abdc-ae1f5fc7826c', '96153093-3cfe-4742-904d-082fda5ae357', '08b7f452-154f-40dd-b211-38c0ab23a43b', '9c891958-ab84-4fde-a1ca-5d78eeaa394f', '1418516c-8077-40a6-9f51-41b9918afc9f', '024d6120-f86f-4e13-8155-9b20e55d99ad', 'c87fb18d-b128-44be-9e86-8ce394233d49', '34b1ff62-2372-473f-b93f-cca96e2eb61a', '56fe2f0a-c17c-4d2a-93ee-b06d2fbd3dff', 'd38d30d2-4da0-4138-97f9-211255fc288b', 'ab228ff2-1100-4100-ab63-88ea99a7e943', '506dd0b4-fee5-4dcc-874e-f0235bfa2828', '26685da3-b9c4-4ed8-a558-2ca25be64085', '7322b9d3-3a26-4da4-a750-af006850174b', '0514918f-5ffa-4e77-b220-8b64830a40a7', 'bf43b8cb-967c-4523-97f3-011be9762178', '7b5d0d66-16ad-4291-8b06-099a356cea4b', 'd5eddf04-7beb-4a20-a045-33d11424d8cc', '90a0a1ba-c8c2-4e00-ab60-42b4d59cc516', '4b414d7a-cbf0-4624-88de-c805138d9469', '6c173c55-d1c9-4285-9b10-e21d678e7999', 'a241e0cd-2994-4d43-93b2-100bf77f7da7', 'c6422c29-1ec6-4b3c-a7dc-d909e4b6c776', 'e17fa03f-00c2-4084-a0f3-87bf51645714', '5809b5c8-d3fb-4995-bbe9-46b35ba7a837', '501088ff-9114-47f3-a7e7-36457bd82fab', 'f936b0ec-fd29-495c-80b9-96023b5fa460', 'd08b5426-3ef3-46b2-ae28-910d7b7937ca', 'f01c43e6-1957-441b-898f-b2c6273ba646', '3b128da7-919a-4a50-a440-f685220695ab', '8818059e-66b1-4ac0-b638-d427ce75bfc4', 'dde81f60-5a15-44f3-b60c-c7b169e554a3', '16a1b6cc-dced-4ab1-b9be-0b664667a67f', '8819cc44-6594-44a9-a97a-6df141cf952d', '040e807f-1b40-4a81-a096-ea383cdec0ed', '56b7a25d-b800-4320-b4f2-aa1668ecf1da', 'd206b9af-4e0c-4a6f-899f-020da44952d7', '53cb9724-e752-4070-bb8e-da16f0be0ba8', '019aa88e-c52c-497c-a823-a89b306889b1', 'f68874ca-c5fd-498e-902c-d872a7d16ee8', '5d2a5b1f-1089-4250-8aaa-f525a55043d6', '04cbcd12-88b1-4650-a755-8ad8499df957', 'dfff59bb-507a-4c4e-b9fa-2cf08b448a44', 'b1995568-c79d-401a-ba73-390016f8a02b', 'a905c316-7ee3-41d2-9b55-6e12c3317b9a', '29a2382d-fddd-49d1-98ee-d0f129b4994f', 'c9b92b8a-b5d3-4d7a-9229-8a450f5ab16a', 'ce5c0cab-9e87-4057-b6df-4045faf73d59', 'af3b6cf3-10d3-4277-8d01-2f4a9d448120', '96dfd8dd-9c48-4fc9-b886-fcbc74b3a6aa', '5ec36386-f181-4891-a12b-6c37896b6bbd', '371b2cc3-9ef5-420f-923b-b6c32920bac0', '80b20778-c6c4-45d4-8710-e55b22534600', '17f91db9-6a80-49d6-be28-b95abf1b8dbe', '4a9b7c39-7855-4c17-80ab-6a3f09c2eff1', 'd835f022-ee2a-43af-999e-cfeccc326ad0', '9cbe8973-fcb1-48b2-944d-35e1183ccb08', 'e4ef434d-8c32-4783-8bfa-4b355dbede8c', '0a44e499-75e2-497c-9343-f1b4262c8ed5', '791c6bdd-fbd5-4b2d-9d9e-7c2c3d0be271', 'b6e46e39-96f2-4857-8b8c-05946f97398f', '68be3455-0d3d-4252-a25d-e98d1f93959c', 'db22c6ea-fbf0-4591-a740-fdb11f104cd6', '5436bb08-3024-4498-8189-60954bb6c170', '0750c8db-f160-4546-bc94-fb7a61cd5001', '6c07253d-cead-49a8-946f-7fe0844137bc', '54855400-8642-44e4-81cc-7075cfea6f37', 'ac26a1e1-5de8-40c3-84b2-3f604cd7e391', 'fae87739-ac61-4ea0-9923-f2bd1b7bcbb5', '2c0509a5-6ec8-4390-ae2d-02ba73bab59e', '2f006589-53f7-411d-8279-359c69fc2b1f', '6f788244-8f9b-46dd-b5e1-b19f3a5e6131', 'c24041ef-b239-40de-b347-df426e0a2073', 'c7e71994-63b8-4fe3-a6ef-f962925aa112', 'c2d5a1a4-760c-4554-9d08-9992f08d35af', 'f78ecdfd-cb09-4f8d-b5e7-6e1d6adf8f88', '1c2563c0-56d0-4848-b2c8-d85691baa0c2', '549c7520-0777-479c-9090-8c83d7ff44fe', '5b423e62-762a-4ab4-b45d-e3081843e829', '4c1cfa51-39f4-4782-8e18-e32bcdaaa296', 'e79a895e-c380-485d-ab80-5072fdd94906', '0cf1c80e-7b02-4407-b59d-9a87f051217b', '77cf495c-34d0-4bca-9533-835c7062484c', '50ba7074-a1f3-4c47-bdca-ebed33745690', '3776d9dd-4495-42e6-96f1-0900b1a651bc', 'd0351fa3-86b9-4e5f-b602-5cc26b2b4e1a', 'ba7a5c70-ef14-4223-940e-d9b65a5c2f44', '0eb15e18-7b3c-4baf-b6ba-49cce461f02d', 'ddea4258-9527-4c8e-aa23-04c9f39d454b', '74088f01-1e96-469c-94e0-f61dc139ab51', '0afd8c89-7ad5-49d8-a85e-39ed10b37b11', '9787f01c-8a0d-4df6-a587-5d30a565f15b', 'fe2952b9-e54c-4fcf-a3f4-0d1b0cdcf532', '128f391d-c159-49cf-a223-c29d146e605c', '2d422992-adc4-4918-8695-1be3ea54fad0', '5443a9e4-61b3-4a58-b622-e0865e818197', '95cb1ba4-475a-4fba-af27-6e299b8693d6', '78a13011-7ede-43d7-9926-d26a847a0367', 'a944dbaa-8f75-4eff-8e48-3600be37c913', '9ba225a5-a1cc-4ebd-bed6-c5b65c881038', 'e0444955-bb69-4d70-9137-acb2a70d80de', 'a1aff3ec-86a5-44d0-b936-91b5f6019078', '2ddf1d36-57da-4007-8272-96c8e3962564', '304386a2-c613-4f16-9327-3f6a2167058b', 'ea28d9ae-a068-4a1d-b763-7020fc76a49a', '9cb0355f-2e29-4d16-876c-0458a685a829', '52be54e4-79c9-469a-807b-0fb6d455b788', '64fc0b3b-d89c-47a4-baae-fc6d5cca946d', '43dfd9d5-6650-4815-8a95-9e55a0457528', 'd736675f-7033-400a-92c7-757bf805b112', '393cafae-c886-463a-8e0e-5138959e1a67', '51883c97-7a92-461f-a1e7-6d0f3ce88999', 'ef917ef5-ddbb-4398-828b-4467007c4b32', '030ac600-cb66-4641-8f22-af782a925ac0', '2a52bd60-903c-4fbe-97ab-4079853853b4', '847d3905-a68a-47cf-9569-f9ff452358dd', 'e631b00c-caa4-40a0-a619-1938e43365ed', '15ee5af7-6e29-4af8-ba31-65da47953658', 'c29771b5-48ee-4d68-9980-793abaf98b19', 'c10f7bd8-9d31-4d23-b0a9-f340ff3aa1d0', 'a49ae307-917e-4aa5-92ed-478f0bccc1d5', '74b6746c-d008-49ce-b752-583ff964da8c', '234a6ada-1f19-4af0-b30b-2715e9d6827c', '22a33e3b-2807-4070-b4c9-548afecb263a', '4d0530d8-d9dc-4311-b635-8903c20f89cb', 'f52103bb-f7e3-4164-a161-ef984538455a', 'bfbd37dc-5a95-4ffe-a366-16c9b9bf557d', '55ec0987-804b-499d-9d94-5fa8c1f94bb7', '204a52cf-b02c-43eb-a468-09c0d3a8bf5e', 'f840c68d-b8d4-411a-add9-79ddb4a5a01d', 'b0535bfe-3132-4a86-92f5-e737403415f1', '8be7bad4-d871-4e0b-b739-c83dba0a6b48', 'ee97aa03-f966-4850-b6fa-245ad3f06746', '52aa95e0-2ad7-461d-a359-3d6f0ba3bf00', '1ff650aa-2505-4f9f-afdc-62affd4a59f3', '491e29c6-4036-4672-9499-7b8d52381b75', '9139c65b-fffd-48de-9b9a-6f3e8ca3b58f', '3c66e192-a190-4d74-b725-52b496ae4e71', 'b397eafa-4896-45c8-88b7-296ab4c72f90', '4cb9b6e3-3826-4e84-8379-1f90b822e929', '90e9e729-d7f6-4641-ae0b-248da1d41c1b', '57f7700e-fa43-4490-afe8-06a649b1edef', 'eee3c5f8-5463-466a-8f2b-962983fda9d6', '6045b6c3-2ae7-499f-99a6-831538329df6', 'c9d6a978-b2f2-47ba-a226-5fba26d893dd', '89b2bca8-bf9e-4af0-9013-e7fdbd1ca74b', '52ee2513-9fcf-44a7-aec9-05cc0408de42', 'b21a3332-ba3c-4abd-ad81-710b1622e68e', '47e84f03-09ae-430d-9abb-d071de9c8f6f', '108d4049-e88b-475e-a248-afeaae2782ef', '5dc7ce26-be22-44be-b7f7-d7195e3d2a88', 'dda8d9e0-ecd1-43bd-aee7-9a9bd16ec428', '4bd1fd10-9ba1-436c-8752-27b7a4560ddb', '4095fe81-5265-4056-bf0d-9dc5f205f721', 'c19bee4f-1b7f-419f-a258-3024a1e24362', '2b1643a6-dfa2-4737-8167-5d2f44345f62', 'c6664015-b8d1-45c3-9e0f-30ed1fcbaace', 'a80b5b94-9818-4a0b-827e-ba7595f24f7d', '7350bb95-48a2-4e12-84e2-5adc4a27aeb1', 'ddc5f6f6-9c3c-4f88-8f6a-0226adcb5919', 'ba30fa04-e616-49f3-9d46-24e1d872b9a4', '3f9ff912-1e91-4605-9249-c40022e202d7', 'd2c30556-8bc6-4dd4-b2fe-c4c3c4ab5885', '6d3d901a-bdc5-47cd-8883-6b4effd4ff7c', 'e3932a7d-dfe7-44ae-bc26-c40afd5f96bf', 'a39488f8-1cda-4fef-92a3-0b397ee5f847', '696c2f1c-a5a2-4245-813e-3c11aa0c33bc', '4045f62d-ccb1-4d3d-bda4-1d85573774ae', 'ca6d2373-81b7-44ac-9383-0a7b44218b4a', '478c3c4a-8272-4ae3-b40c-3768035edbae', '40633562-4688-40d3-96ca-174c7e57eff2', 'ba1e7fed-7979-4496-9b2b-a296c1905013', 'f293b8f8-d4b6-42f2-bdbc-c596e6b8c9d0', '6e34871f-b153-4397-9f96-f967f99a340a', 'a4a9a24c-9bf6-4b4c-b27d-7928040a6be1', 'ab940563-ed69-49c3-9432-9dd0242c38dc', 'b704f1a5-a183-440d-9620-3a5eb025e872', '8760b3e3-dd7e-47c4-84f7-e6ffc9ed2e28', 'b1913a3c-939a-4179-8747-8894039bb9d2', 'b8bddd94-c115-48e8-836e-d2c0dd86b4c1', 'c81493e0-1680-4dc1-9ea9-342b2c6a2c3d', '120da6e4-646a-4e93-8c48-633ecb6b5463', '5b5148bf-3080-497e-bd0f-ee4e0f043ce7', '92fffecd-d047-4272-93f0-e34fe1095654', '1d12cf0b-ce2c-4a35-ba08-b3c207d5e833', 'b23e5e4f-2045-4010-a8cf-b08f78bb8a7f', '3576ff00-12fe-467d-b0b3-70c17e6fa7fb', 'c60b0ae9-3caa-4f4f-9fea-bc87ec18db01', 'fdd72e16-c00d-4f61-9064-fdab83067032', '7830c781-83c6-4a39-ab31-12e93e49971c', 'ef33333f-13b6-49b1-a93a-7e1814c1579c', '1dfe49de-acd2-4751-a309-010e4c80746e', 'd2667211-fce8-46e3-bb93-9fb6f96e2862', '2ec55b7b-20b7-4cf9-b1af-7eebc566c5fd', 'd29ed45d-0ac5-495e-b8f8-0a3f5bca1d68', 'a64c69a2-30d1-43d1-8168-276cdccf9e9a', '20067308-9771-4a1c-98cf-0897c25af809', '43d796a8-4572-4d85-9070-c0e2e2a12e83', 'fb3ef62d-77fc-4235-a38a-1cb6a8f5988b', '7a09516b-1eb1-497e-a6c2-2cbe33bcdaf1', 'a9dca19f-c4e5-423f-b3dd-42b299f03a05', '3775bd14-c4a6-451c-8b65-a80427853797', '100a2224-3dec-4029-b3b2-83910acd126d', '3878a9bc-f6d7-420e-b7cc-1c01a5b88a7d', '1f99ff9c-e148-4911-a841-72fc13a6715c', 'e7627f2d-71ea-47af-9726-506d71cb7008', 'b4a105c3-b28f-4ff4-bbf0-d077055c4047', 'b2064e5c-3e87-4d5b-bf26-f72f6a7f3057', '8bffddb8-9f0d-4702-97f8-260774038c37', 'f99db900-76dd-4e80-ad56-1b5ba62ed0b6', 'e4ceda65-fe1e-4e04-bc5b-095ef9d287ab', 'a06b53a5-5d79-453f-acee-6acc4233380d', 'b3bdb06f-cbba-45ca-b220-9ff15c0a70b3', '9e17b192-16d3-4e67-94d9-f60c52b202b2', 'd7216c28-bc23-4e22-8070-7ca66d10fbaa', '2abb3181-da0b-43ab-84b4-b05f9a71006c', '0783acfc-6542-4e0d-95dc-688beee0abaa', 'adb284a2-bad5-4764-b598-4ab9d292e393', '4e56ac00-e29a-425d-97dd-76a0eca4185c', '731a9eaf-948e-49c0-b104-291038854e3d', '1fc924be-e382-4bd0-b044-98cb6337cacf', 'd1051d18-d9e9-4cfb-9d34-f68b4f116570', '14ddd434-dce1-4c75-a4b6-ab9074d0eab2', '50ec2195-9903-46fa-b266-69167a128a5b', '200bbbcf-9a01-41b5-8b06-954e88759a42', 'ab8310f9-d92e-42e4-83d5-f04b8a826418', '6c9ebcca-bff1-479e-bb26-217cad961da4', '544d33c7-2d7b-4ceb-8cc6-bbfacada9ba9', 'df0a6bfc-a117-4ec8-b2b8-7da7eccd33d8', '31086c8b-ca13-444e-b1aa-e37ecde17fab', '82fccaa9-2976-405b-bc44-444cff9c2815', '6dff6735-c2c5-46b0-8a41-69167bf2fdf7', '5dcdcebb-f642-4dce-a91e-c5a5cc7295bc', '34c6808c-3d1b-4940-bad9-194e18819726', '914b7b3d-7c0f-4f3b-a1d9-38f22996809c', 'b4d9941c-dbf9-4f20-a7fe-bc18cdd88f46', 'ec479475-c53c-4241-8214-59f5cbb0cba0', '2e52f6c4-8141-45a1-96bc-b013a4f4250a', '3ea47f07-37cd-4213-8b55-0987189f3fb5', '98f8dbec-5823-4e7b-b1da-4a0a66dd1841', '6ac1b426-5073-4f20-a6d7-1223585831f1', '3a68d0db-c832-4dfa-9990-900d2ae4052e', 'dc24ea82-98be-49f1-abd8-30a387a82c69', '7b2d625e-8cc2-42fb-9e29-b34aa426c58c', '505644f1-8bcf-443c-910e-56529d51d2ea', '615c020d-21f5-4546-a5ee-fd44d03d85fc', '1d58f4e4-9bea-4bdd-b2c8-cd8f0d9c453b', '63513359-8a62-411b-befc-832b186e26cd', '7d06e6b0-c3f6-4ae1-8ca9-b8caa6855a5d', '85cdf949-7be1-4b5f-92e1-e3727eb8fe94', '387d0ce1-8293-4dce-9768-a1c209033753', 'a0ad096f-db7f-4482-8cc2-7351d09f0e73', 'fce2e2ed-a8d1-4aec-adcf-7994c0652e5b', '3f6b4f1f-7647-4366-a686-c9e305a630c2', '30851f37-ca48-46ec-9eef-a58f074d08ef', '8a0c460a-3834-4fc6-a8d6-e9c8543d36ab', '28b73b0f-d5b1-4281-8525-9fb24f4b217c', '0aea1091-b42a-4c8a-a204-27c189cc5ed8', '6bd99aa1-0b1d-415d-af9c-94f28f1a0042', '919e9c5d-58f7-4390-a195-3a9f9a5171ed', '42600244-505c-4e82-be7b-426daa23b7fd', '9fc4c493-901a-4947-860e-43db9656684b', 'f61d1915-5424-4fb8-9846-ad37acfd3909', '1f2318ea-b550-4abf-b342-a78c067ce3af', '4695169d-6637-40db-b9c3-a14133da2bad', '55f38b46-4ad2-41c6-9723-a684ad69e88d', '456a3a88-691e-4539-9ecf-0f80d149e8c8', '43dec340-cc37-4cf6-8ed3-e1d0f1c61fc4', 'dbcbb0d3-532c-4074-9392-4551fc760b23', 'a04a033f-037a-434a-bd51-e902c27d7cf4'} {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  769.134532] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Segment QoS.
[  769.152343] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Segment QoS taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 122 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.152569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.152685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.152990] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.153134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.153433] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Segment QoS Is:0.
[  769.153433] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Segment QoS Are:set() {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  769.153433] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Members).
[  769.417207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  769.417207] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  769.417207] env[60890]: ERROR oslo_messaging.rpc.server 
[  769.694320] env[60890]: INFO networking_nsxv3.common.synchronization [req-6284ee71-a7ce-4c5f-98aa-6fc71a0faaf1 req-f771bb22-982b-4a44-b8e6-c052e2403cc3 tempest-ServersTestFqdnHostnames-535084876 tempest-ServersTestFqdnHostnames-535084876-project-member] Enqueued Resource ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b with Priority: HIGHEST for action security_group_members
[  769.694320] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b with Priority: HIGHEST for action security_group_members
[  769.695012] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.695012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.695012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.695012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.695012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.695308] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-337aaaf2-15f5-46a8-8b59-236cfe63466b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.695308] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.695308] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.695444] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.695444] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.695554] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.695604] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.695715] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.695792] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.695878] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('337aaaf2-15f5-46a8-8b59-236cfe63466b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  769.696143] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b08e31b0958d4b3b9539591a305273e3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  769.706533] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b08e31b0958d4b3b9539591a305273e3
[  769.956194] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.956403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.956551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.956812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.956962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.957246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.957404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.957525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.957744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.958002] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.958162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.958282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.958496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.958674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.958834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.959090] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.959276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.959358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.959567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.959703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  769.960173] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  769.960392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  769.960517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  769.960731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  769.960873] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  769.961107] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 419eac73860f4f999d20b1d8935f5e93 in queue reply_156db6c4256a4c58884afce15e6f5340
[  769.970076] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 419eac73860f4f999d20b1d8935f5e93
[  770.202143] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  770.202287] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  770.202354] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  770.202581] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  770.202751] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  770.202954] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  770.203051] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  770.203111] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  770.203285] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  770.203392] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  770.203493] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 337aaaf2-15f5-46a8-8b59-236cfe63466b already deleted.
[  770.203578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-337aaaf2-15f5-46a8-8b59-236cfe63466b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  770.463169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  770.464318] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  770.464318] env[60890]: ERROR oslo_messaging.rpc.server 
[  771.000446] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  771.000650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  771.000777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  771.001054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  771.001203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  771.001461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  771.001621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  771.001741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  771.001963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  771.002204] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  771.002359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  771.002476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  771.002681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  771.002834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  771.002973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  771.003220] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  771.003374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  771.003493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  771.003701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  771.003838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  771.004090] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  771.004246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  771.004364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  771.004572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  771.004708] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  771.004970] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg dcdae1d1f298493ba78f15c637164506 in queue reply_156db6c4256a4c58884afce15e6f5340
[  771.012140] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dcdae1d1f298493ba78f15c637164506
[  771.507530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  771.508069] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  771.508069] env[60890]: ERROR oslo_messaging.rpc.server 
[  771.941516] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 125 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  771.941659] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  771.941731] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  771.941946] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  772.116934] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  772.122293] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Security Group (Members) Is:0.
[  772.122385] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Security Group (Members) Are:set() {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[  772.122572] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Inventory metadata have been refreshed.
[  772.122645] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Realizing 20/96 resources of Type:Ports
[  772.122744] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 519bac90-d40c-494c-8eb9-6c53e239f10a with Priority: HIGH for action port
[  772.122839] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 32332759-ca87-4ef6-955a-325ad700200d with Priority: HIGH for action port
[  772.122908] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 03fdd0ff-044d-4309-905b-d51a3bab1b81 with Priority: HIGH for action port
[  772.122969] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 55f117f0-ba76-491f-b5c5-7441d3ff7787 with Priority: HIGH for action port
[  772.123029] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 9435dc4a-ead0-4afd-a099-668739ae29f4 with Priority: HIGH for action port
[  772.123088] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 66d98031-6842-491c-9961-c1b4d0fd5d53 with Priority: HIGH for action port
[  772.123146] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 0f875a82-d52b-4617-86ef-09c6a8aeccbb with Priority: HIGH for action port
[  772.123205] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 2126120d-06ba-46a5-a74c-77f3b18345ab with Priority: HIGH for action port
[  772.123263] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 17b157f7-bf15-4a5c-a271-9af72764322d with Priority: HIGH for action port
[  772.123321] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 474946e7-99a3-4290-959e-f2dc1645b999 with Priority: HIGH for action port
[  772.123379] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 4fb6c40c-bf99-4b55-9f6b-ca451ded5287 with Priority: HIGH for action port
[  772.123437] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 7e152039-3f45-45d1-b2a1-c7acb717f68b with Priority: HIGH for action port
[  772.123495] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 1ca72486-d800-43c5-907b-725bb958b3a3 with Priority: HIGH for action port
[  772.123552] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 2a1bc88a-7641-4017-874c-f15ee8ec3621 with Priority: HIGH for action port
[  772.123609] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: d357f4a7-7fe1-4677-a32d-b2a6c5780592 with Priority: HIGH for action port
[  772.123667] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 99c860bb-14da-4e95-b94e-d77d15224b7a with Priority: HIGH for action port
[  772.123724] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: fb2c7956-257f-4d1d-8dbb-83b5350798c3 with Priority: HIGH for action port
[  772.123782] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: c4585efa-40da-4b2f-81a8-523efec14826 with Priority: HIGH for action port
[  772.123840] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 7e3eabc0-1383-412e-a875-c4e73daf6808 with Priority: HIGH for action port
[  772.123898] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: f599de71-cb96-446c-b853-bfcdc671260e with Priority: HIGH for action port
[  772.124032] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  772.126466] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  772.126638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  772.126757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  772.127048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  772.127206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  772.127466] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  772.127615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  772.127726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  772.127944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  772.128192] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  772.128337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  772.128447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  772.128642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  772.128772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  772.128911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  772.129174] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  772.129340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  772.129452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  772.129653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  772.129779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  772.130007] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  772.130148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  772.130259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  772.130454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  772.130580] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  772.130825] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 484fd5dea0814454a11d9432ca47ad52 in queue reply_156db6c4256a4c58884afce15e6f5340
[  772.136232] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 484fd5dea0814454a11d9432ca47ad52
[  772.632825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  772.633438] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  772.633438] env[60890]: ERROR oslo_messaging.rpc.server 
[  773.090469] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  773.090700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  773.090832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  773.091103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  773.091250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  773.091511] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-488 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  773.091666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  773.091785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  773.092013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  773.092265] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  773.092421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  773.092538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  773.092768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  773.092924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  773.093104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-488" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  773.093417] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  773.093574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  773.093833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  773.094067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  773.094210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  773.094443] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-052e0a67-2acf-4b39-b214-1b06b4d5db08 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  773.094594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  773.094712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  773.095108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  773.095265] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('052e0a67-2acf-4b39-b214-1b06b4d5db08',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  773.095502] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6c533df1ce214a56af6290fcde6f9262 in queue reply_156db6c4256a4c58884afce15e6f5340
[  773.100882] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6c533df1ce214a56af6290fcde6f9262
[  773.598074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-052e0a67-2acf-4b39-b214-1b06b4d5db08" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  773.598558] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  773.598558] env[60890]: ERROR oslo_messaging.rpc.server 
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [req-e427e7c0-2dce-4861-a095-2ac7835ce821 req-4edb7df9-c58d-4631-81e7-11674b93b777 tempest-AttachVolumeShelveTestJSON-2132759461 tempest-AttachVolumeShelveTestJSON-2132759461-project-member] Enqueued Resource ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f with Priority: HIGHEST for action security_group_members
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f with Priority: HIGHEST for action security_group_members
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 519bac90-d40c-494c-8eb9-6c53e239f10a with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 03fdd0ff-044d-4309-905b-d51a3bab1b81 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0f875a82-d52b-4617-86ef-09c6a8aeccbb with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d357f4a7-7fe1-4677-a32d-b2a6c5780592 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f599de71-cb96-446c-b853-bfcdc671260e with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7e3eabc0-1383-412e-a875-c4e73daf6808 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c4585efa-40da-4b2f-81a8-523efec14826 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: fb2c7956-257f-4d1d-8dbb-83b5350798c3 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2a1bc88a-7641-4017-874c-f15ee8ec3621 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 99c860bb-14da-4e95-b94e-d77d15224b7a with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 1ca72486-d800-43c5-907b-725bb958b3a3 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7e152039-3f45-45d1-b2a1-c7acb717f68b with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 4fb6c40c-bf99-4b55-9f6b-ca451ded5287 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 474946e7-99a3-4290-959e-f2dc1645b999 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 17b157f7-bf15-4a5c-a271-9af72764322d with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 66d98031-6842-491c-9961-c1b4d0fd5d53 with Priority: HIGH for action port
[  774.450404] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2126120d-06ba-46a5-a74c-77f3b18345ab with Priority: HIGH for action port
[  774.455697] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9435dc4a-ead0-4afd-a099-668739ae29f4 with Priority: HIGH for action port
[  774.455697] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 32332759-ca87-4ef6-955a-325ad700200d with Priority: HIGH for action port
[  774.455697] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 55f117f0-ba76-491f-b5c5-7441d3ff7787 with Priority: HIGH for action port
[  774.455697] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.455697] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.455697] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.455697] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.455697] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8377ea59c99a48a4bf0f782d2ced9e4f in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.455697] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.455697] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-519bac90-d40c-494c-8eb9-6c53e239f10a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.455697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.455697] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('519bac90-d40c-494c-8eb9-6c53e239f10a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.455697] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e1d02ee43a3c442389d26189f9efe81a in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.457056] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.457056] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.457056] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.457056] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.457241] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.457291] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-03fdd0ff-044d-4309-905b-d51a3bab1b81 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.457426] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.457563] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.457829] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.457969] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03fdd0ff-044d-4309-905b-d51a3bab1b81',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.458217] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 91ca3ed3df4a48beaa1c1c3d0f7326b3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.459912] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.460060] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.460112] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.460251] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.460342] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.460474] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-0f875a82-d52b-4617-86ef-09c6a8aeccbb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.460598] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.460656] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.460812] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.460893] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f875a82-d52b-4617-86ef-09c6a8aeccbb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.461143] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d456f114a63c4024be2e34e75e845652 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.462977] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.462977] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.462977] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.462977] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.462977] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.463188] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-d357f4a7-7fe1-4677-a32d-b2a6c5780592 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.463188] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.463188] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.463344] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.463498] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d357f4a7-7fe1-4677-a32d-b2a6c5780592',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.463532] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0533be9286c34d76ba260f2a33fb3ad4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.464199] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.464323] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.464375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.464517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.464595] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.464738] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-f599de71-cb96-446c-b853-bfcdc671260e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.464853] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.464918] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.465121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.465121] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f599de71-cb96-446c-b853-bfcdc671260e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.465265] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 62ab8690b3804c0a95a918eff07bf041 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.465985] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.466264] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.466344] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.466504] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.466617] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.466768] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-7e3eabc0-1383-412e-a875-c4e73daf6808 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.466875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.466931] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.467092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.467639] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e3eabc0-1383-412e-a875-c4e73daf6808',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.467639] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0fa3a6381a6b4663b5b39635cf790e89 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.468008] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.468114] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.468184] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.468326] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.468403] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.468543] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-c4585efa-40da-4b2f-81a8-523efec14826 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.468636] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.468691] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.468831] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.468902] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4585efa-40da-4b2f-81a8-523efec14826',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.469046] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ea0a3ee19648422090f7f540e6646555 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.469820] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.469820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.469940] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.470006] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.470061] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.470203] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-fb2c7956-257f-4d1d-8dbb-83b5350798c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.470294] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.471288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.471288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.471288] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fb2c7956-257f-4d1d-8dbb-83b5350798c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.471288] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a6c500ab19ee42d381048d289ab147c9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.471288] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.471460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.471460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.471608] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.472630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.472630] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2a1bc88a-7641-4017-874c-f15ee8ec3621 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.472630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.472630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.472630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.472630] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2a1bc88a-7641-4017-874c-f15ee8ec3621',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.473175] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 28e733a650604dadbb4da9c535309da9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.473508] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.473598] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.473679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.473884] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.473945] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.474124] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-99c860bb-14da-4e95-b94e-d77d15224b7a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.474258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.474297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.474469] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.474569] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('99c860bb-14da-4e95-b94e-d77d15224b7a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.474726] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7fca241f49484f018ef7ca7e3f6466ff in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.476382] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.476382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.476382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.476382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.476382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.476382] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-1ca72486-d800-43c5-907b-725bb958b3a3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.476382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.476382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.476698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.476698] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1ca72486-d800-43c5-907b-725bb958b3a3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.476698] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg cd4c6c04a3bb45138494c3570bc263b2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.477916] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.477916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.477916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.477916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.477916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.477916] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-7e152039-3f45-45d1-b2a1-c7acb717f68b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.477916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.477916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.478220] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.478220] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e152039-3f45-45d1-b2a1-c7acb717f68b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.478294] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 29ae186c56ba403f9f30265d74e77c7f in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.478962] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.479075] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.479134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.479283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.479354] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.480431] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.480431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.480431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.480431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.480431] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4fb6c40c-bf99-4b55-9f6b-ca451ded5287',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.480431] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c1028d23a0e844ec867f4971a8909e44 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.480708] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.480708] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.480868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.480933] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.480965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.481090] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-474946e7-99a3-4290-959e-f2dc1645b999 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.481225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.481225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.481361] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.481434] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('474946e7-99a3-4290-959e-f2dc1645b999',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.481572] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 32d6130f3c534bd99c947a7894e7b66a in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.483024] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.483024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.483024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.483024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.483024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.483024] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-17b157f7-bf15-4a5c-a271-9af72764322d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.483024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.483024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.483342] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.484985] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17b157f7-bf15-4a5c-a271-9af72764322d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.484985] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3f9db9cf3e6642eba29a870b8322e48c in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.485815] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.485815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.485815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.485815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.485815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.485815] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-66d98031-6842-491c-9961-c1b4d0fd5d53 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.485815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.486951] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.486951] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.486951] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('66d98031-6842-491c-9961-c1b4d0fd5d53',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.486951] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 58fc907d9829431582faeacbf90e5b6b in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.487926] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.487926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.487926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.487926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.487926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.487926] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2126120d-06ba-46a5-a74c-77f3b18345ab taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.487926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.487926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.488789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.488789] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2126120d-06ba-46a5-a74c-77f3b18345ab',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.488789] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3d80b7baf43c45ed8de2de324010bcfd in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.489262] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.489262] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.489262] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.489262] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.489262] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.489650] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-9435dc4a-ead0-4afd-a099-668739ae29f4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.489650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.489650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.489650] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.489813] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9435dc4a-ead0-4afd-a099-668739ae29f4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.489868] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b999da2959794454ba70d7e97d03138a in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.490570] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.490699] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.490788] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.490996] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.491098] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.491257] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-32332759-ca87-4ef6-955a-325ad700200d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.491585] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.491585] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.491585] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.491585] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32332759-ca87-4ef6-955a-325ad700200d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.491829] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0879c045f50d4264b751170ff07a3288 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.492677] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.492798] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.492890] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.493080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.493164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.493324] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-55f117f0-ba76-491f-b5c5-7441d3ff7787 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  774.493397] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  774.493458] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  774.493610] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  774.493689] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('55f117f0-ba76-491f-b5c5-7441d3ff7787',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  774.493848] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0e0950c97e23433f9fa277bfc710f572 in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.496979] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8377ea59c99a48a4bf0f782d2ced9e4f
[  774.497992] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e1d02ee43a3c442389d26189f9efe81a
[  774.497992] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 91ca3ed3df4a48beaa1c1c3d0f7326b3
[  774.498302] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d456f114a63c4024be2e34e75e845652
[  774.498724] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0533be9286c34d76ba260f2a33fb3ad4
[  774.499145] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 62ab8690b3804c0a95a918eff07bf041
[  774.503725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0fa3a6381a6b4663b5b39635cf790e89
[  774.503725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ea0a3ee19648422090f7f540e6646555
[  774.503725] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.503725] env[60890]: Traceback (most recent call last):
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/poll.py", line 111, in wait
[  774.503725] env[60890]:     listener.cb(fileno)
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.503725] env[60890]:     result = function(*args, **kwargs)
[  774.503725] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.503725] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.503725] env[60890]:     return method(*args, **kwargs)
[  774.503725] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.503725] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.503725] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.503725] env[60890]:     result = self.transport._send(
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.503725] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.503725] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.503725] env[60890]:     raise result
[  774.503725] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.503725] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.503725] env[60890]: Removing descriptor: 52
[  774.503725] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.503725] env[60890]: Traceback (most recent call last):
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/poll.py", line 111, in wait
[  774.503725] env[60890]:     listener.cb(fileno)
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.503725] env[60890]:     result = function(*args, **kwargs)
[  774.503725] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.503725] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.503725] env[60890]:     return method(*args, **kwargs)
[  774.503725] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.503725] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.503725] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.503725] env[60890]:     result = self.transport._send(
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.503725] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.503725] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.503725] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.503725] env[60890]:     raise result
[  774.503725] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.503725] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.503725] env[60890]: Removing descriptor: 32
[  774.503725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ce8d43b51f40444c9dadfad4e4df34cc in queue reply_156db6c4256a4c58884afce15e6f5340
[  774.510946] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a6c500ab19ee42d381048d289ab147c9
[  774.511521] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 28e733a650604dadbb4da9c535309da9
[  774.511963] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7fca241f49484f018ef7ca7e3f6466ff
[  774.512410] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cd4c6c04a3bb45138494c3570bc263b2
[  774.520520] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 29ae186c56ba403f9f30265d74e77c7f
[  774.520520] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c1028d23a0e844ec867f4971a8909e44
[  774.520520] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 32d6130f3c534bd99c947a7894e7b66a
[  774.520520] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ce8d43b51f40444c9dadfad4e4df34cc
[  774.524568] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3f9db9cf3e6642eba29a870b8322e48c
[  774.524568] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 58fc907d9829431582faeacbf90e5b6b
[  774.526544] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3d80b7baf43c45ed8de2de324010bcfd
[  774.528140] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b999da2959794454ba70d7e97d03138a
[  774.536809] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0879c045f50d4264b751170ff07a3288
[  774.536809] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0e0950c97e23433f9fa277bfc710f572
[  774.960537] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.960799] env[60890]: Traceback (most recent call last):
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.960847] env[60890]:     timer()
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.960847] env[60890]:     cb(*args, **kw)
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.960847] env[60890]:     result = function(*args, **kwargs)
[  774.960847] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.960847] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.960847] env[60890]:     return method(*args, **kwargs)
[  774.960847] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.960847] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.960847] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.960847] env[60890]:     result = self.transport._send(
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.960847] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.960847] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.960847] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.960847] env[60890]:     raise result
[  774.961448] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.961448] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.963444] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.964040] env[60890]: Traceback (most recent call last):
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.964040] env[60890]:     timer()
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.964040] env[60890]:     cb(*args, **kw)
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.964040] env[60890]:     result = function(*args, **kwargs)
[  774.964040] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.964040] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.964040] env[60890]:     return method(*args, **kwargs)
[  774.964040] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.964040] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.964040] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.964040] env[60890]:     result = self.transport._send(
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.964040] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.964040] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.964040] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.964040] env[60890]:     raise result
[  774.964040] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.964040] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.995247] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.995469] env[60890]: Traceback (most recent call last):
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.995504] env[60890]:     timer()
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.995504] env[60890]:     cb(*args, **kw)
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.995504] env[60890]:     result = function(*args, **kwargs)
[  774.995504] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.995504] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.995504] env[60890]:     return method(*args, **kwargs)
[  774.995504] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.995504] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.995504] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.995504] env[60890]:     result = self.transport._send(
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.995504] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.995504] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.995504] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.995504] env[60890]:     raise result
[  774.996143] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.996143] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.996143] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.996143] env[60890]: Traceback (most recent call last):
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.996143] env[60890]:     timer()
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.996143] env[60890]:     cb(*args, **kw)
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.996143] env[60890]:     result = function(*args, **kwargs)
[  774.996143] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.996143] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.996143] env[60890]:     return method(*args, **kwargs)
[  774.996143] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.996143] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.996143] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.996143] env[60890]:     result = self.transport._send(
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.996143] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.996143] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.996143] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.996143] env[60890]:     raise result
[  774.996143] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.996143] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.996885] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.996885] env[60890]: Traceback (most recent call last):
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.996885] env[60890]:     timer()
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.996885] env[60890]:     cb(*args, **kw)
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.996885] env[60890]:     result = function(*args, **kwargs)
[  774.996885] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.996885] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.996885] env[60890]:     return method(*args, **kwargs)
[  774.996885] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.996885] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.996885] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.996885] env[60890]:     result = self.transport._send(
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.996885] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.996885] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.996885] env[60890]:     raise result
[  774.996885] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.996885] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.996885] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.996885] env[60890]: Traceback (most recent call last):
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.996885] env[60890]:     timer()
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.996885] env[60890]:     cb(*args, **kw)
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.996885] env[60890]:     result = function(*args, **kwargs)
[  774.996885] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.996885] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.996885] env[60890]:     return method(*args, **kwargs)
[  774.996885] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.996885] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.996885] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.996885] env[60890]:     result = self.transport._send(
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.996885] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.996885] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.996885] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.996885] env[60890]:     raise result
[  774.996885] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.996885] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.998332] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.998332] env[60890]: Traceback (most recent call last):
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.998332] env[60890]:     timer()
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.998332] env[60890]:     cb(*args, **kw)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.998332] env[60890]:     result = function(*args, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.998332] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.998332] env[60890]:     return method(*args, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.998332] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.998332] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.998332] env[60890]:     result = self.transport._send(
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.998332] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.998332] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.998332] env[60890]:     raise result
[  774.998332] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.998332] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.998332] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.998332] env[60890]: Traceback (most recent call last):
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.998332] env[60890]:     timer()
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.998332] env[60890]:     cb(*args, **kw)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.998332] env[60890]:     result = function(*args, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.998332] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.998332] env[60890]:     return method(*args, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.998332] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.998332] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.998332] env[60890]:     result = self.transport._send(
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.998332] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.998332] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.998332] env[60890]:     raise result
[  774.998332] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.998332] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.998332] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  774.998332] env[60890]: Traceback (most recent call last):
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  774.998332] env[60890]:     timer()
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  774.998332] env[60890]:     cb(*args, **kw)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  774.998332] env[60890]:     result = function(*args, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  774.998332] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  774.998332] env[60890]:     return method(*args, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  774.998332] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  774.998332] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  774.998332] env[60890]:     result = self.transport._send(
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  774.998332] env[60890]:     return self._driver.send(target, ctxt, message,
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  774.998332] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  774.998332] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  774.998332] env[60890]:     raise result
[  774.998332] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  774.998332] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  774.998332] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.000686] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.000686] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f already deleted.
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b7ab88b7-dbf1-4f23-a0e2-819a39b7b72f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.000686] env[60890]: Traceback (most recent call last):
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.000686] env[60890]:     timer()
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.000686] env[60890]:     cb(*args, **kw)
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.000686] env[60890]:     result = function(*args, **kwargs)
[  775.000686] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.000686] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.000686] env[60890]:     return method(*args, **kwargs)
[  775.000686] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.000686] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.000686] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.000686] env[60890]:     result = self.transport._send(
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.000686] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.000686] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.000686] env[60890]:     raise result
[  775.000686] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.000686] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.000686] env[60890]: Traceback (most recent call last):
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.000686] env[60890]:     timer()
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.000686] env[60890]:     cb(*args, **kw)
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.000686] env[60890]:     result = function(*args, **kwargs)
[  775.000686] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.000686] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.000686] env[60890]:     return method(*args, **kwargs)
[  775.000686] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.000686] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.000686] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.000686] env[60890]:     result = self.transport._send(
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.000686] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.000686] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.000686] env[60890]:     raise result
[  775.000686] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.000686] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  775.000686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.000686] env[60890]: Traceback (most recent call last):
[  775.000686] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.000686] env[60890]:     timer()
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.002965] env[60890]:     cb(*args, **kw)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.002965] env[60890]:     result = function(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.002965] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.002965] env[60890]:     return method(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.002965] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.002965] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.002965] env[60890]:     result = self.transport._send(
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.002965] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.002965] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.002965] env[60890]:     raise result
[  775.002965] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.002965] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  775.002965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.002965] env[60890]: Traceback (most recent call last):
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.002965] env[60890]:     timer()
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.002965] env[60890]:     cb(*args, **kw)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.002965] env[60890]:     result = function(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.002965] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.002965] env[60890]:     return method(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.002965] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.002965] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.002965] env[60890]:     result = self.transport._send(
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.002965] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.002965] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.002965] env[60890]:     raise result
[  775.002965] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.002965] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  775.002965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.002965] env[60890]: Traceback (most recent call last):
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.002965] env[60890]:     timer()
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.002965] env[60890]:     cb(*args, **kw)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.002965] env[60890]:     result = function(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.002965] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.002965] env[60890]:     return method(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.002965] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.002965] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.002965] env[60890]:     result = self.transport._send(
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.002965] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.002965] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.002965] env[60890]:     raise result
[  775.002965] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.002965] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  775.002965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.002965] env[60890]: Traceback (most recent call last):
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.002965] env[60890]:     timer()
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.002965] env[60890]:     cb(*args, **kw)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.002965] env[60890]:     result = function(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.002965] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.002965] env[60890]:     return method(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.002965] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.002965] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.002965] env[60890]:     result = self.transport._send(
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.002965] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.002965] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.002965] env[60890]:     raise result
[  775.002965] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.002965] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  775.002965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.002965] env[60890]: Traceback (most recent call last):
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.002965] env[60890]:     timer()
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.002965] env[60890]:     cb(*args, **kw)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.002965] env[60890]:     result = function(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.002965] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.002965] env[60890]:     return method(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.002965] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.002965] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.002965] env[60890]:     result = self.transport._send(
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.002965] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.002965] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.002965] env[60890]:     raise result
[  775.002965] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.002965] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  775.002965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.002965] env[60890]: Traceback (most recent call last):
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.002965] env[60890]:     timer()
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.002965] env[60890]:     cb(*args, **kw)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.002965] env[60890]:     result = function(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.002965] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.002965] env[60890]:     return method(*args, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.002965] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.002965] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.002965] env[60890]:     result = self.transport._send(
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.002965] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.002965] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.002965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.002965] env[60890]:     raise result
[  775.002965] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.002965] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  775.014448] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  775.014448] env[60890]: Traceback (most recent call last):
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  775.014448] env[60890]:     timer()
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  775.014448] env[60890]:     cb(*args, **kw)
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  775.014448] env[60890]:     result = function(*args, **kwargs)
[  775.014448] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  775.014448] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  775.014448] env[60890]:     return method(*args, **kwargs)
[  775.014448] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  775.014448] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  775.014448] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  775.014448] env[60890]:     result = self.transport._send(
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  775.014448] env[60890]:     return self._driver.send(target, ctxt, message,
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  775.014448] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  775.014448] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  775.014448] env[60890]:     raise result
[  775.014448] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  775.014448] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  778.893379] env[60890]: INFO networking_nsxv3.common.synchronization [req-f383332a-2dc9-46fc-bd83-cde413c20499 req-8a19472a-c07a-4cb9-8de3-efa2b49c7644 tempest-ServersNegativeTestMultiTenantJSON-777735062 tempest-ServersNegativeTestMultiTenantJSON-777735062-project-member] Enqueued Resource ID: 73652d40-4acc-4b61-bbb7-756981ddd98a with Priority: HIGHEST for action security_group_members
[  778.893379] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 73652d40-4acc-4b61-bbb7-756981ddd98a with Priority: HIGHEST for action security_group_members
[  778.893379] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  778.893379] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-73652d40-4acc-4b61-bbb7-756981ddd98a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-73652d40-4acc-4b61-bbb7-756981ddd98a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-73652d40-4acc-4b61-bbb7-756981ddd98a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-73652d40-4acc-4b61-bbb7-756981ddd98a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  778.893379] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  778.893379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  778.893379] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('73652d40-4acc-4b61-bbb7-756981ddd98a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  778.893379] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2e3ca5cdb59d4640aa7b6922614994ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  778.901310] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2e3ca5cdb59d4640aa7b6922614994ad
[  779.203584] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  779.203798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  779.203934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  779.204280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  779.204434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  779.205263] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  779.205263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  779.205263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  779.205559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  779.205773] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  779.205959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  779.206083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  779.206306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  779.206457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  779.206611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  779.206869] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  779.207030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  779.207266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  779.207484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  779.207624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  779.207870] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  779.208068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  779.208585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  779.208585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  779.208585] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  779.208819] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg dbb4130b264a4de3be4e1c338055f989 in queue reply_156db6c4256a4c58884afce15e6f5340
[  779.217202] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dbb4130b264a4de3be4e1c338055f989
[  779.394471] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  779.394643] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  779.394696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  779.395000] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  779.395090] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  779.395288] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  779.395389] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  779.395446] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  779.395601] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  779.395682] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  779.395767] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 73652d40-4acc-4b61-bbb7-756981ddd98a already deleted.
[  779.395866] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-73652d40-4acc-4b61-bbb7-756981ddd98a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  779.711403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  779.711950] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  779.711950] env[60890]: ERROR oslo_messaging.rpc.server 
[  780.254186] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  780.254464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  780.254548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  780.254854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  780.255025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  780.255292] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  780.255505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  780.255570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  780.255802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  780.256133] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  780.256298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  780.256417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  780.256630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  780.256776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  780.256927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  780.257195] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  780.257349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  780.257469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  780.257678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  780.257820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  780.258143] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  780.258310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  780.258432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  780.258646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  780.258786] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  780.259018] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 23912ec32b9c42899b6d5e7f31fe586a in queue reply_156db6c4256a4c58884afce15e6f5340
[  780.265311] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 23912ec32b9c42899b6d5e7f31fe586a
[  780.761994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  780.762535] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  780.762535] env[60890]: ERROR oslo_messaging.rpc.server 
[  781.304456] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  781.304456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  781.304850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  781.304898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  781.305066] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  781.305331] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  781.305493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  781.305616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  781.305861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  781.306121] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  781.306276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  781.306396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  781.306609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  781.306859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  781.306923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  781.307154] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  781.307311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  781.307429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  781.307758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  781.307920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  781.308214] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  781.308369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  781.308488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  781.308699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  781.308907] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  781.309072] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6b96259ec5554b199fe508c90cffc696 in queue reply_156db6c4256a4c58884afce15e6f5340
[  781.314470] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6b96259ec5554b199fe508c90cffc696
[  781.811061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  781.811579] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  781.811579] env[60890]: ERROR oslo_messaging.rpc.server 
[  782.353802] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  782.354092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  782.354140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  782.354412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  782.354556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  782.354865] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  782.355040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  782.355163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  782.355381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  782.355624] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  782.355774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  782.355886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  782.356105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  782.356247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  782.356414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  782.356640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  782.356789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  782.356902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  782.357107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  782.357235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  782.357468] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  782.357617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  782.357731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  782.357945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  782.358073] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  782.358302] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg da2ce52c0d77413b8361785af1bc1459 in queue reply_156db6c4256a4c58884afce15e6f5340
[  782.363251] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg da2ce52c0d77413b8361785af1bc1459
[  782.862551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  782.862551] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  782.862551] env[60890]: ERROR oslo_messaging.rpc.server 
[  783.029114] env[60890]: INFO networking_nsxv3.common.synchronization [req-7ecfde97-ca57-4305-8519-f082bfabeeaf req-d1dcc29e-78f7-4e14-9167-67be62beb3a5 tempest-ServerAddressesNegativeTestJSON-257136632 tempest-ServerAddressesNegativeTestJSON-257136632-project-member] Enqueued Resource ID: 5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627 with Priority: HIGHEST for action security_group_members
[  783.029623] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627 with Priority: HIGHEST for action security_group_members
[  783.029967] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.030114] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.030208] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.030465] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.030536] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.030723] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.030851] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.030923] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.031156] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.031329] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.031435] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.031497] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.031644] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.031727] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.031823] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  783.032014] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg fe3f3631c5184e719dff3713ee22ca68 in queue reply_156db6c4256a4c58884afce15e6f5340
[  783.043072] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fe3f3631c5184e719dff3713ee22ca68
[  783.317301] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.317503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.317630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.317896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.318044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.318300] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.318485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.318610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.318836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.319083] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.319243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.319360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.319563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.319706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.319858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.320302] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.320475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.320597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.320810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.320957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.321176] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.321329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.321455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.321714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.321851] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  783.322084] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4890d5fbdb9e4f8eb7a54a5bb40d2f99 in queue reply_156db6c4256a4c58884afce15e6f5340
[  783.330730] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4890d5fbdb9e4f8eb7a54a5bb40d2f99
[  783.401077] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.401388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.401451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.401677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.401822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.402107] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.402238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.403183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.403183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.403183] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.403183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.403183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.403416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.403416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.403554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.403805] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.403965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.404116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.404306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.404437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.404673] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.404898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.405013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.405225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.405362] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  783.405590] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1dece74a6f9842219a2450a247b19bcf in queue reply_156db6c4256a4c58884afce15e6f5340
[  783.411625] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1dece74a6f9842219a2450a247b19bcf
[  783.539984] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.540261] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.540357] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.540627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.540730] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.540933] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  783.541033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  783.541092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  783.541246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  783.541325] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.541409] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627 already deleted.
[  783.541496] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.824291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  783.824945] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  783.824945] env[60890]: ERROR oslo_messaging.rpc.server 
[  783.908172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  783.908688] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  783.908688] env[60890]: ERROR oslo_messaging.rpc.server 
[  784.665185] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.665473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.665604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.665874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.665951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.666171] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.666330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.666449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.666851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.666918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.667057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.667177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.667386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.667534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.667882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.667960] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.668090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.668211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.668425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.668562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.668804] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.669142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.669142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.669302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.669410] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  784.670282] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 21601cfb190541ca9c8a15ba67b846c5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  784.671486] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.671661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.671785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.672021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.672173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.672415] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.672572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.672901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.672942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.673166] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.673325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.673443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.674603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.674603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.674603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.674603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.674603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.674603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.674603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.674879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.674998] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  784.675156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  784.675275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  784.675482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  784.675613] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  784.675868] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8389ff7d83324ad88c22551e84313b96 in queue reply_156db6c4256a4c58884afce15e6f5340
[  784.676762] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 21601cfb190541ca9c8a15ba67b846c5
[  784.677328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  784.677859] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  784.677859] env[60890]: ERROR oslo_messaging.rpc.server 
[  784.680047] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8389ff7d83324ad88c22551e84313b96
[  785.180668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  785.181203] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  785.181203] env[60890]: ERROR oslo_messaging.rpc.server 
[  785.338325] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.338626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.338812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.339709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.340118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.340838] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.341220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.341521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.342875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.342875] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.342875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.342875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.346654] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.346654] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.346654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.346654] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  785.346654] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7148deae21604791850bc5dde898ee43 in queue reply_156db6c4256a4c58884afce15e6f5340
[  785.352237] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7148deae21604791850bc5dde898ee43
[  785.534199] env[60890]: INFO networking_nsxv3.common.synchronization [req-37fc2bc5-3ab6-499b-ada7-49988a7232ba req-3fdc33aa-5226-4df2-a1ed-45569e780412 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  785.534901] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  785.535277] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.535426] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.535498] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.535779] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.535869] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.536498] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.536498] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.536498] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.536498] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.536898] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.536898] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.537037] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.537974] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.537974] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.537974] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  785.537974] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 29126c0faf254df3b57d8ca89ad37483 in queue reply_156db6c4256a4c58884afce15e6f5340
[  785.548649] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 29126c0faf254df3b57d8ca89ad37483
[  785.789169] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.789169] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.789169] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.789169] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.789169] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  785.789169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  785.789169] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  785.789169] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 98c2ca9d68104a5188498166d6b9051e in queue reply_156db6c4256a4c58884afce15e6f5340
[  785.794212] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 98c2ca9d68104a5188498166d6b9051e
[  785.850042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  785.850042] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  785.850042] env[60890]: ERROR oslo_messaging.rpc.server 
[  786.018518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.018518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.018518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.018518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.018518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.018518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.018518] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  786.018518] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6c0e1dc5a587495d8a07cf9800e3570e in queue reply_156db6c4256a4c58884afce15e6f5340
[  786.023166] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6c0e1dc5a587495d8a07cf9800e3570e
[  786.031415] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.031415] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.031415] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.031415] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.031415] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.031415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.031415] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  786.037556] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a8ba875fdec84748b12335f10fd681f3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  786.037556] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a8ba875fdec84748b12335f10fd681f3
[  786.040995] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.040995] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.040995] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  786.040995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.290764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  786.291293] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  786.291293] env[60890]: ERROR oslo_messaging.rpc.server 
[  786.519923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  786.520747] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  786.520747] env[60890]: ERROR oslo_messaging.rpc.server 
[  786.538090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  786.538657] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  786.538657] env[60890]: ERROR oslo_messaging.rpc.server 
[  786.833230] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.833552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.833552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.833826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.833972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.834229] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.834386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.834505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.835263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.835603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.835677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.835807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.836046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.836202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.836354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.836609] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.836780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.836897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.837172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.837325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  786.837560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  786.837716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  786.837838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  786.838048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  786.838185] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  786.838415] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7a0b74ff777345699402253c1673910e in queue reply_156db6c4256a4c58884afce15e6f5340
[  786.843050] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7a0b74ff777345699402253c1673910e
[  787.083327] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.083531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.083660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.083930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.084094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.084361] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.084519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.084641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.084888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.085140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.085298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.085418] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.085626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.085776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.085927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.086178] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.086335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.086455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.086661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.086798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.087519] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.087635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.087683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.087892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.088037] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  787.088320] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 07ca2a46ff654d178cfa85c2ae97ea94 in queue reply_156db6c4256a4c58884afce15e6f5340
[  787.093385] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 07ca2a46ff654d178cfa85c2ae97ea94
[  787.096270] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.096448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.096572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.096798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.096938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.097208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.097521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.097653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.097867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.098110] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.098267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.098386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.098590] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.098734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.098879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.099125] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.099279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.099397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.099602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.099736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.099971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.100153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.100273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.100480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.100612] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  787.100832] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9248a233635740569cabbec1a7cedf4c in queue reply_156db6c4256a4c58884afce15e6f5340
[  787.105128] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9248a233635740569cabbec1a7cedf4c
[  787.340976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  787.341503] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  787.341503] env[60890]: ERROR oslo_messaging.rpc.server 
[  787.591873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  787.592687] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  787.592687] env[60890]: ERROR oslo_messaging.rpc.server 
[  787.603303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  787.603789] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  787.603789] env[60890]: ERROR oslo_messaging.rpc.server 
[  787.883540] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.883540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.883540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.883917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.883917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.884202] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.884355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.884471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.884688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.885131] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.885306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.885445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.885653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.885894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.886126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.886400] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.886560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.886679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.886930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.887073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  787.887317] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  787.887467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  787.887583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  787.887794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  787.887957] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  787.888289] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3125d1218b904212a2a34f78522e53b4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  787.894246] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3125d1218b904212a2a34f78522e53b4
[  788.164246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.164449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.164579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.164912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.165069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.165325] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.165483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.165602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.165831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.166070] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.166239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.166335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.166538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.166677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.166824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.167079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.167246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.167365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.167569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.167705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.167939] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.168107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.168225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.168429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.168565] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  788.168793] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1c4672878abd48a2b0f0b2b4d4b3f191 in queue reply_156db6c4256a4c58884afce15e6f5340
[  788.170706] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.170883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.171008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.171220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.171358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.171610] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.171764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.171883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.172104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.173163] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.173163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.173163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.173163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.173311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.173382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.173632] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.173816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.173930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.174135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.174271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.174509] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.174666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.174816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.175028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.175162] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  788.175378] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2ef21758e27a4c94b6a46487805c1c21 in queue reply_156db6c4256a4c58884afce15e6f5340
[  788.176249] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1c4672878abd48a2b0f0b2b4d4b3f191
[  788.176789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  788.177228] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  788.177228] env[60890]: ERROR oslo_messaging.rpc.server 
[  788.179316] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2ef21758e27a4c94b6a46487805c1c21
[  788.391477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  788.392018] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  788.392018] env[60890]: ERROR oslo_messaging.rpc.server 
[  788.679602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  788.680241] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  788.680241] env[60890]: ERROR oslo_messaging.rpc.server 
[  788.717430] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.717640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.717773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.718050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.718206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.718464] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.718623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.718745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.718977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.719226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.719581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.719735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.719957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.720123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.720281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.720536] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.720694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.720815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.721031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.721221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.721459] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.721615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.721736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.721947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.722090] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  788.722327] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 30fb513a86544a7bb2e2da57646b4a95 in queue reply_156db6c4256a4c58884afce15e6f5340
[  788.727828] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 30fb513a86544a7bb2e2da57646b4a95
[  788.936334] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.936632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.936667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.936915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.937096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.937323] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-254 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.937479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.937600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.937817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.938060] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.938215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.938360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.938594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.938753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.938893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-254" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.939193] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.939637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.939780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.940136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.940299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  788.940544] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-780a67bc-1479-45af-b8d6-38dcdc7ff0be taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  788.940705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  788.940825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  788.941037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  788.941174] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('780a67bc-1479-45af-b8d6-38dcdc7ff0be',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  788.941413] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 602a26a23c41448382ea62f04175bb0d in queue reply_156db6c4256a4c58884afce15e6f5340
[  788.946747] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 602a26a23c41448382ea62f04175bb0d
[  789.224419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  789.225056] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  789.225056] env[60890]: ERROR oslo_messaging.rpc.server 
[  789.232523] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.232739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.232868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.233138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.233284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.233537] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.233785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.233930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.234184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.234434] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.234594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.234713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.234976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.235121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.235269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.235539] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.235710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.235894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.236128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.236271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.236504] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.236661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.236781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.236994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.237160] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  789.237395] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7f4a6e4cb2d649669351c841a570f0e3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  789.242020] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7f4a6e4cb2d649669351c841a570f0e3
[  789.443238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-780a67bc-1479-45af-b8d6-38dcdc7ff0be" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  789.443811] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  789.443811] env[60890]: ERROR oslo_messaging.rpc.server 
[  789.739658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  789.740196] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  789.740196] env[60890]: ERROR oslo_messaging.rpc.server 
[  789.772821] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.773021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.773149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.773419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.773555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.773811] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.773976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.774078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.774290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.774621] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.774823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.774955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.775164] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.775304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.775451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.775701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.775852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.775966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.776188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.776317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  789.776583] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  789.776734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  789.776850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  789.777054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  789.777207] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  789.777437] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d44374dcae50428085586a54687d59be in queue reply_156db6c4256a4c58884afce15e6f5340
[  789.782369] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d44374dcae50428085586a54687d59be
[  790.239310] env[60890]: INFO networking_nsxv3.common.synchronization [req-f383332a-2dc9-46fc-bd83-cde413c20499 req-4737a38a-0694-43dc-bef3-ae0e299d670e tempest-ServersNegativeTestMultiTenantJSON-777735062 tempest-ServersNegativeTestMultiTenantJSON-777735062-project-member] Enqueued Resource ID: 73652d40-4acc-4b61-bbb7-756981ddd98a with Priority: HIGHEST for action security_group_members
[  790.239935] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 73652d40-4acc-4b61-bbb7-756981ddd98a with Priority: HIGHEST for action security_group_members
[  790.240330] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.240458] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.240561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.241506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.241506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.241506] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-73652d40-4acc-4b61-bbb7-756981ddd98a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.241506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-73652d40-4acc-4b61-bbb7-756981ddd98a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.241506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-73652d40-4acc-4b61-bbb7-756981ddd98a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.241506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-73652d40-4acc-4b61-bbb7-756981ddd98a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.241506] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.241506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.241506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.242049] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.242049] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.242049] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('73652d40-4acc-4b61-bbb7-756981ddd98a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  790.242049] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f0f2c92742a34101afc0d578dc912da1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  790.262292] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f0f2c92742a34101afc0d578dc912da1
[  790.279475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  790.279970] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  790.279970] env[60890]: ERROR oslo_messaging.rpc.server 
[  790.307241] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.307441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.307563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.307834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.307976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.308249] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.308413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.308591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.308858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.309234] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.309454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.309610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.309870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.310059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.310249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.310518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.310721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.310880] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.311149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.311331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.311616] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.311824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.311993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.312281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.312455] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  790.312750] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1680d89bf9fd49da91c6df4620362e30 in queue reply_156db6c4256a4c58884afce15e6f5340
[  790.317703] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1680d89bf9fd49da91c6df4620362e30
[  790.744215] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.744372] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.744446] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.744659] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.744755] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.745000] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  790.745123] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  790.745211] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  790.745396] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  790.745484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.745596] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 73652d40-4acc-4b61-bbb7-756981ddd98a already deleted.
[  790.745703] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-73652d40-4acc-4b61-bbb7-756981ddd98a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.815142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  790.815639] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  790.815639] env[60890]: ERROR oslo_messaging.rpc.server 
[  791.000342] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.000536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.000684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.000960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.001298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.001649] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.001792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.001909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.002140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.002381] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.002533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.002669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.003131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.003299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.003455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.003704] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.003857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.003973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.004200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.004333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.004644] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.004806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.004934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.005144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.005272] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  791.005501] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8d03a9d5e4504051a6a11d833a52275f in queue reply_156db6c4256a4c58884afce15e6f5340
[  791.008336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.008512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.008640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.008861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.009710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.009710] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.010040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.010218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.010538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.010636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.010821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.010945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.011129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.011351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.011520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.011771] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.011928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.012060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.012272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.012408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.012642] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.012819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.012949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.013260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.013409] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  791.013624] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 24d2440771b745108a7f5c779b7bc0bb in queue reply_156db6c4256a4c58884afce15e6f5340
[  791.014500] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8d03a9d5e4504051a6a11d833a52275f
[  791.032166] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 24d2440771b745108a7f5c779b7bc0bb
[  791.507735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  791.508269] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  791.508269] env[60890]: ERROR oslo_messaging.rpc.server 
[  791.516970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  791.517416] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  791.517416] env[60890]: ERROR oslo_messaging.rpc.server 
[  791.670205] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.670408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.670539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.670812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.670957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.671216] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-658 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.671371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.671489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.671703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.672274] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.672490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.672615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.672855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.673007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.673159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-658" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.673411] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.673564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.673841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.674135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.674437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.674577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-52653254-9225-4860-8dd1-2f7bc2ac9857 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.674832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.674970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.675184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.675320] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52653254-9225-4860-8dd1-2f7bc2ac9857',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  791.675553] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0d2cfdd5b0e64557a817182fd117ae3a in queue reply_156db6c4256a4c58884afce15e6f5340
[  791.677706] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.677869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.677991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.678205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.678470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.678573] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.678722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.678846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.679049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.679281] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.680266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.680266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.680266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.680266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.680266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.680266] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.680449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.680733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.684417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.684634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  791.684928] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  791.685099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  791.685224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  791.685445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  791.685582] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  791.685841] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8860b951092048a7a591933a813fa3c3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  791.687251] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0d2cfdd5b0e64557a817182fd117ae3a
[  791.691546] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8860b951092048a7a591933a813fa3c3
[  792.188134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-52653254-9225-4860-8dd1-2f7bc2ac9857" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  792.188865] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  792.188865] env[60890]: ERROR oslo_messaging.rpc.server 
[  792.190302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  792.190464] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  792.190464] env[60890]: ERROR oslo_messaging.rpc.server 
[  793.144639] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.145005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.145291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.145377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.145473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.145736] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.145895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.146017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.146232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.146581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.146753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.146877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.147131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.147265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.147469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.147746] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.148171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.148606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.148940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.149815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.150072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.150295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.150490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.150807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.151012] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  793.152377] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c74cf12362694b87b2274489f6c532f8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  793.156816] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c74cf12362694b87b2274489f6c532f8
[  793.400136] env[60890]: INFO networking_nsxv3.common.synchronization [req-7ecfde97-ca57-4305-8519-f082bfabeeaf req-f3bba3eb-e7f5-40d9-a963-8b18f942a5cf tempest-ServerAddressesNegativeTestJSON-257136632 tempest-ServerAddressesNegativeTestJSON-257136632-project-member] Enqueued Resource ID: 5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627 with Priority: HIGHEST for action security_group_members
[  793.400668] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627 with Priority: HIGHEST for action security_group_members
[  793.401012] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.401106] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.401170] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.401363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.401443] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.401588] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.401731] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.401731] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.401873] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.402004] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.402089] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.402144] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.402274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.402347] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.402432] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  793.402605] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ca4487c0320c4c8eac33a2b1a47c276c in queue reply_156db6c4256a4c58884afce15e6f5340
[  793.416760] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ca4487c0320c4c8eac33a2b1a47c276c
[  793.654136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  793.654691] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  793.654691] env[60890]: ERROR oslo_messaging.rpc.server 
[  793.904096] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.904248] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.904317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.904532] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.904621] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.904828] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  793.904931] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  793.904993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  793.905153] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  793.905228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  793.905311] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627 already deleted.
[  793.905386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-5e8be9a0-7481-4e6c-bcc9-1dbdab2ee627" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  794.204200] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  794.204556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  794.204734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  794.205062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  794.205210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  794.205474] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  794.205633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  794.205754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  794.205996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  794.206240] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  794.206397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  794.206516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  794.206724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  794.206868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  794.207019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  794.207266] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  794.207421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  794.207537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  794.207742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  794.212555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  794.212704] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  794.212947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  794.213073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  794.213280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  794.213411] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  794.213648] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 067eaaa34fa24ebba50f65fd073e7f3c in queue reply_156db6c4256a4c58884afce15e6f5340
[  794.224214] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 067eaaa34fa24ebba50f65fd073e7f3c
[  794.682494] env[60890]: INFO networking_nsxv3.common.synchronization [None req-036560b7-e8f7-4118-ba3c-3cac8636e872 None None] Enqueued Resource ID: fc80f59a-759a-4009-9334-0ebc678f93a9 with Priority: HIGHEST for action port
[  794.682786] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: fc80f59a-759a-4009-9334-0ebc678f93a9 with Priority: HIGHEST for action port
[  794.683396] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  794.683515] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  794.683591] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  794.683800] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  794.683868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  794.684034] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-fc80f59a-759a-4009-9334-0ebc678f93a9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  794.684127] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-fc80f59a-759a-4009-9334-0ebc678f93a9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  794.684229] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-fc80f59a-759a-4009-9334-0ebc678f93a9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  794.684495] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-fc80f59a-759a-4009-9334-0ebc678f93a9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  794.684611] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fc80f59a-759a-4009-9334-0ebc678f93a9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  794.684891] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5563d80cb4534412b0d6c2788b34c83d in queue reply_156db6c4256a4c58884afce15e6f5340
[  794.699241] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5563d80cb4534412b0d6c2788b34c83d
[  794.716068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  794.716598] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  794.716598] env[60890]: ERROR oslo_messaging.rpc.server 
[  795.190717] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-fc80f59a-759a-4009-9334-0ebc678f93a9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  795.190960] env[60890]: Traceback (most recent call last):
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  795.191016] env[60890]:     timer()
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  795.191016] env[60890]:     cb(*args, **kw)
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  795.191016] env[60890]:     result = function(*args, **kwargs)
[  795.191016] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  795.191016] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  795.191016] env[60890]:     return method(*args, **kwargs)
[  795.191016] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  795.191016] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  795.191016] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  795.191016] env[60890]:     result = self.transport._send(
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  795.191016] env[60890]:     return self._driver.send(target, ctxt, message,
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  795.191016] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  795.191016] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  795.191016] env[60890]:     raise result
[  795.191016] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  795.191016] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  795.317741] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  795.318031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  795.318078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  795.318333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  795.318488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  795.318749] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  795.318908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  795.319028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  795.319250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  795.319476] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  795.319631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  795.319750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  795.319958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  795.320118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  795.320273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  795.320510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  795.320665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  795.320785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  795.320993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  795.321126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  795.321365] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-eb89ad6c-3b0a-4823-abdc-688b384131b3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  795.321519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  795.321635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  795.321837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  795.321974] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('eb89ad6c-3b0a-4823-abdc-688b384131b3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  795.322202] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 82a93db060ef4c9995866935fce01d26 in queue reply_156db6c4256a4c58884afce15e6f5340
[  795.329569] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 82a93db060ef4c9995866935fce01d26
[  795.825031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-eb89ad6c-3b0a-4823-abdc-688b384131b3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  795.825525] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  795.825525] env[60890]: ERROR oslo_messaging.rpc.server 
[  796.469954] env[60890]: INFO networking_nsxv3.common.synchronization [None req-1b3a0acb-ecba-442f-98ea-ebdd10b9b169 None None] Enqueued Resource ID: 066bfa5f-2f15-4229-8ef7-d546a244333b with Priority: HIGHEST for action port
[  796.470223] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 066bfa5f-2f15-4229-8ef7-d546a244333b with Priority: HIGHEST for action port
[  796.470818] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  796.470946] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  796.471010] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  796.471206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  796.471285] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  796.471426] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-066bfa5f-2f15-4229-8ef7-d546a244333b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  796.471514] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-066bfa5f-2f15-4229-8ef7-d546a244333b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  796.471643] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-066bfa5f-2f15-4229-8ef7-d546a244333b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  796.471766] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-066bfa5f-2f15-4229-8ef7-d546a244333b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  796.471842] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('066bfa5f-2f15-4229-8ef7-d546a244333b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  796.472057] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 848378d9addf46559e22e6c6b5b27934 in queue reply_156db6c4256a4c58884afce15e6f5340
[  796.478956] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 848378d9addf46559e22e6c6b5b27934
[  796.868791] env[60890]: INFO networking_nsxv3.common.synchronization [req-37fc2bc5-3ab6-499b-ada7-49988a7232ba req-04a2e987-db0f-420a-8bf8-f2c425232121 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  796.869316] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  796.869663] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  796.869779] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  796.869852] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  796.870044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  796.870130] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  796.870289] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  796.870387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  796.870450] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  796.870598] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  796.870744] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  796.870840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  796.870898] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  796.871197] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  796.871273] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  796.871417] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  796.871546] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c39ffec826a0442da39d62344cf01066 in queue reply_156db6c4256a4c58884afce15e6f5340
[  796.879801] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c39ffec826a0442da39d62344cf01066
[  796.974433] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-066bfa5f-2f15-4229-8ef7-d546a244333b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  796.974675] env[60890]: Traceback (most recent call last):
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  796.974734] env[60890]:     timer()
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  796.974734] env[60890]:     cb(*args, **kw)
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  796.974734] env[60890]:     result = function(*args, **kwargs)
[  796.974734] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  796.974734] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  796.974734] env[60890]:     return method(*args, **kwargs)
[  796.974734] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  796.974734] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  796.974734] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  796.974734] env[60890]:     result = self.transport._send(
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  796.974734] env[60890]:     return self._driver.send(target, ctxt, message,
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  796.974734] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  796.974734] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  796.974734] env[60890]:     raise result
[  796.974734] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  796.974734] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  797.373225] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  797.373388] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  797.373464] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  797.373681] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  797.373777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  797.373977] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  797.374076] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  797.374139] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  797.374283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  797.374373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  797.374458] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  797.374538] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  798.989292] env[60890]: INFO networking_nsxv3.common.synchronization [req-b111699a-7ed5-4eb8-93b3-afaf56ab7f0b req-0d734a93-a569-4981-84c2-8ae6ba379eee tempest-ServerActionsV293TestJSON-1045119592 tempest-ServerActionsV293TestJSON-1045119592-project-member] Enqueued Resource ID: e717cb10-dc75-40d9-a823-032d7367566c with Priority: HIGHEST for action security_group_members
[  798.989734] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e717cb10-dc75-40d9-a823-032d7367566c with Priority: HIGHEST for action security_group_members
[  798.990065] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  798.990192] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  798.990260] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  798.990451] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  798.990536] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  798.990686] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e717cb10-dc75-40d9-a823-032d7367566c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  798.990785] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  798.990849] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  798.991037] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  798.991179] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  798.991272] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  798.991352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  798.991472] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  798.991547] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  798.991638] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e717cb10-dc75-40d9-a823-032d7367566c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  798.991818] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c8e787619b6a4f57910c731f345e655c in queue reply_156db6c4256a4c58884afce15e6f5340
[  799.000971] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c8e787619b6a4f57910c731f345e655c
[  799.391101] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  799.391315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  799.391445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  799.391721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  799.391876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  799.392177] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  799.392339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  799.392462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  799.392739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  799.392992] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  799.393151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  799.393270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  799.393480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  799.393627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  799.393781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  799.394042] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  799.394223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  799.394347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  799.394558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  799.394693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  799.394978] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  799.395135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  799.395256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  799.395504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  799.395644] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  799.395885] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 69bc22d356f646cb80d492085e55ffac in queue reply_156db6c4256a4c58884afce15e6f5340
[  799.410134] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 69bc22d356f646cb80d492085e55ffac
[  799.496658] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  799.496658] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  799.496658] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e717cb10-dc75-40d9-a823-032d7367566c already deleted.
[  799.496658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  799.897978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  799.898540] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  799.898540] env[60890]: ERROR oslo_messaging.rpc.server 
[  800.497019] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  800.497341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  800.497410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  800.497662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  800.497825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  800.498070] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  800.498228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  800.498350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  800.498571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  800.498826] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  800.499031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  800.499181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  800.499374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  800.499523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  800.499667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  800.499923] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  800.500362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  800.500362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  800.500535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  800.500589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  800.500789] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  800.500947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  800.501066] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  800.501278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  800.501454] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  800.501644] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 30579deca987482e92541586102bf7c7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  800.509536] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 30579deca987482e92541586102bf7c7
[  801.003705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  801.004249] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  801.004249] env[60890]: ERROR oslo_messaging.rpc.server 
[  801.649686] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  801.649686] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  801.649686] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  801.649686] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  801.649686] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  801.649686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  801.649686] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  801.649686] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4609eb74087249bea20a09281ee3ed36 in queue reply_156db6c4256a4c58884afce15e6f5340
[  801.651922] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4609eb74087249bea20a09281ee3ed36
[  802.148485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  802.149367] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  802.149367] env[60890]: ERROR oslo_messaging.rpc.server 
[  802.541811] env[60890]: INFO networking_nsxv3.common.synchronization [req-6969668b-1d63-41ce-90ab-0ca814391d4d req-91b123ae-1860-4b51-bcc7-186b60addccf tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  802.541811] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  802.541811] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.541811] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.541811] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.541811] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.541811] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  802.541811] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7e90b76de2774659b088f4ca90a41a53 in queue reply_156db6c4256a4c58884afce15e6f5340
[  802.607296] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7e90b76de2774659b088f4ca90a41a53
[  802.728409] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.728409] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.728409] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.728409] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.728409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.729460] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.729460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.729460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.729746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.729901] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  802.730168] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 60596ca6c75248c3b5a5b75151464e84 in queue reply_156db6c4256a4c58884afce15e6f5340
[  802.744035] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 60596ca6c75248c3b5a5b75151464e84
[  802.872114] env[60890]: INFO networking_nsxv3.common.synchronization [req-6969668b-1d63-41ce-90ab-0ca814391d4d req-f2015708-fefd-4575-a072-baf828a8740f tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  802.872640] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  802.872979] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.873101] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  802.873168] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  802.873361] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  802.873496] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  802.873618] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  802.873702] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.042107] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.042270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.042317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.042527] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.042624] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.042891] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.042996] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.043053] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.043212] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.043292] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.043379] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  803.043455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.043600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.043742] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.043879] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.043971] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.044044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.044182] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.044257] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.044347] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  803.044539] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a62dc24898e847fcb07bcafd101fb8cb in queue reply_156db6c4256a4c58884afce15e6f5340
[  803.052419] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a62dc24898e847fcb07bcafd101fb8cb
[  803.111824] env[60890]: INFO networking_nsxv3.common.synchronization [None req-9e4c6c43-986c-473e-8730-ea4111cd0c20 None None] Enqueued Resource ID: 52daac1c-afa2-4feb-9408-563d8deef2b6 with Priority: HIGHEST for action port
[  803.112457] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 52daac1c-afa2-4feb-9408-563d8deef2b6 with Priority: HIGHEST for action port
[  803.112902] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.113167] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.113328] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.113598] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.113782] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.114041] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-52daac1c-afa2-4feb-9408-563d8deef2b6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.114274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-52daac1c-afa2-4feb-9408-563d8deef2b6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.114426] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-52daac1c-afa2-4feb-9408-563d8deef2b6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.114717] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-52daac1c-afa2-4feb-9408-563d8deef2b6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.114931] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('52daac1c-afa2-4feb-9408-563d8deef2b6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  803.115223] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c14f7216da7242048019644b02994221 in queue reply_156db6c4256a4c58884afce15e6f5340
[  803.120682] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c14f7216da7242048019644b02994221
[  803.233696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  803.233696] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  803.233696] env[60890]: ERROR oslo_messaging.rpc.server 
[  803.287692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.287692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.287692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.287692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.287692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.287692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.287692] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  803.287692] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 66192a47c6ab4e41a10a33027c726236 in queue reply_156db6c4256a4c58884afce15e6f5340
[  803.292381] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 66192a47c6ab4e41a10a33027c726236
[  803.547140] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.547320] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.547401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.547714] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.547827] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.548088] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.548256] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.548364] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.548568] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.548652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.548741] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  803.548821] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.617831] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-52daac1c-afa2-4feb-9408-563d8deef2b6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.618045] env[60890]: Traceback (most recent call last):
[  803.618045] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  803.618045] env[60890]:     timer()
[  803.618045] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  803.618045] env[60890]:     cb(*args, **kw)
[  803.618045] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  803.618045] env[60890]:     result = function(*args, **kwargs)
[  803.618045] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  803.618045] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  803.618045] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  803.618045] env[60890]:     return method(*args, **kwargs)
[  803.618045] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  803.618045] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  803.618045] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  803.618045] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  803.618045] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  803.618045] env[60890]:     result = self.transport._send(
[  803.618564] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  803.618564] env[60890]:     return self._driver.send(target, ctxt, message,
[  803.618564] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  803.618564] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  803.618564] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  803.618564] env[60890]:     raise result
[  803.618564] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  803.618564] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  803.790512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  803.791017] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  803.791017] env[60890]: ERROR oslo_messaging.rpc.server 
[  803.839948] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.840178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.840320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.840625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.840782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.841051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.841213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.841338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.841561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.841809] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.841970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.842092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.842303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.842447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.842597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.842990] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.843155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.843278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.843493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.843802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  803.843934] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  803.844037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  803.844150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  803.844362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  803.844500] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  803.844737] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 474921aab0c3453fb38ccd63d423ebdd in queue reply_156db6c4256a4c58884afce15e6f5340
[  803.850675] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 474921aab0c3453fb38ccd63d423ebdd
[  804.348095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  804.348095] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  804.348095] env[60890]: ERROR oslo_messaging.rpc.server 
[  804.349559] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.349660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.349709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.350501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.350501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.350501] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.350501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.350912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.350912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.351228] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.351320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.351376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.351589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.351726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.351870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.352234] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.352405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.352522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.352730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.352863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.353101] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.353247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.353361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.353563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.353696] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  804.353920] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e946ac5a09a04b52b783784524bfb21b in queue reply_156db6c4256a4c58884afce15e6f5340
[  804.392158] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e946ac5a09a04b52b783784524bfb21b
[  804.505227] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0e1a9305777746f18f0a3eb5d5b280c4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  804.522012] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0e1a9305777746f18f0a3eb5d5b280c4
[  804.688768] env[60890]: INFO networking_nsxv3.common.synchronization [req-768981e8-8509-4f96-9e5e-46d73f09920b req-257ad4ae-7fe8-4b6c-bbd1-eead5006a9df tempest-SecurityGroupsTestJSON-806588483 tempest-SecurityGroupsTestJSON-806588483-project-member] Enqueued Resource ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 with Priority: HIGHEST for action security_group_members
[  804.688768] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 with Priority: HIGHEST for action security_group_members
[  804.688768] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.688768] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.688768] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.688768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.688768] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('384e1ef2-f71b-4a37-9e25-4fafe57f50e8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  804.688768] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c2887b9dc2de4852b0396991f75fff70 in queue reply_156db6c4256a4c58884afce15e6f5340
[  804.707570] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c2887b9dc2de4852b0396991f75fff70
[  804.857326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  804.857326] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  804.857326] env[60890]: ERROR oslo_messaging.rpc.server 
[  804.891012] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.891012] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.891012] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.891012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.894956] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.894956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.894956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.894956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.894956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.894956] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.894956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.894956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.894956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.894956] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  804.894956] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 709871dde19d43c1bd8ed2f05efc4477 in queue reply_156db6c4256a4c58884afce15e6f5340
[  804.899979] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 709871dde19d43c1bd8ed2f05efc4477
[  804.976280] env[60890]: INFO networking_nsxv3.common.synchronization [None req-2f60561a-7486-4a0d-8024-fec256cdfb63 None None] Enqueued Resource ID: 82565306-6c2d-4eca-9f32-00c2def11dd7 with Priority: HIGHEST for action port
[  804.976280] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 82565306-6c2d-4eca-9f32-00c2def11dd7 with Priority: HIGHEST for action port
[  804.976280] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.976280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.976280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.976280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.976280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  804.976280] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-82565306-6c2d-4eca-9f32-00c2def11dd7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  804.976280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-82565306-6c2d-4eca-9f32-00c2def11dd7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  804.976280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-82565306-6c2d-4eca-9f32-00c2def11dd7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  804.976280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-82565306-6c2d-4eca-9f32-00c2def11dd7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  804.976280] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('82565306-6c2d-4eca-9f32-00c2def11dd7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  804.976280] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg daf6f280d0cc4b2b955f916030d95e61 in queue reply_156db6c4256a4c58884afce15e6f5340
[  804.989082] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg daf6f280d0cc4b2b955f916030d95e61
[  805.191436] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.191627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.191675] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.191883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.191978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.192198] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.192301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.192362] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.192515] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.192596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.192750] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 already deleted.
[  805.192837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.262061] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.262272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.262401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.262698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.262868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.263133] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.263295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.263417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.263634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.263879] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.264128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.264204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.264413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.264558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.264704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.265068] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.265251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.265372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.265584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.265722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.265964] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.266122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.266240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.266504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.266642] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  805.266873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c71888175d0c4d078b4fad2b71777431 in queue reply_156db6c4256a4c58884afce15e6f5340
[  805.274560] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c71888175d0c4d078b4fad2b71777431
[  805.396260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  805.396787] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  805.396787] env[60890]: ERROR oslo_messaging.rpc.server 
[  805.418475] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.418675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.418809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.419083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.419224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.419478] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.419631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.419739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.419948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.423730] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.423730] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.423730] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.423730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.423730] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  805.423730] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 51f00d5b9f8b4d0b8a5490d406c160c7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  805.431498] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 51f00d5b9f8b4d0b8a5490d406c160c7
[  805.479137] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-82565306-6c2d-4eca-9f32-00c2def11dd7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.479137] env[60890]: Traceback (most recent call last):
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  805.479137] env[60890]:     timer()
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  805.479137] env[60890]:     cb(*args, **kw)
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  805.479137] env[60890]:     result = function(*args, **kwargs)
[  805.479137] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  805.479137] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  805.479137] env[60890]:     return method(*args, **kwargs)
[  805.479137] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  805.479137] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  805.479137] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  805.479137] env[60890]:     result = self.transport._send(
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  805.479137] env[60890]:     return self._driver.send(target, ctxt, message,
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  805.479137] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  805.479137] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  805.479137] env[60890]:     raise result
[  805.479137] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  805.479137] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  805.772107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  805.772644] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  805.772644] env[60890]: ERROR oslo_messaging.rpc.server 
[  805.926736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  805.926736] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  805.926736] env[60890]: ERROR oslo_messaging.rpc.server 
[  805.936484] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.936682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.936813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.937123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.937279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.937545] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.937707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.937830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.938051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.938298] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.938457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.938577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.938785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.938932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.939082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.939334] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.939493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.939613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.939819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.939957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  805.940223] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  805.940382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  805.940499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  805.940710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  805.940846] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  805.941076] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3ea94dad925a42779d16f4d393ddb141 in queue reply_156db6c4256a4c58884afce15e6f5340
[  805.946135] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3ea94dad925a42779d16f4d393ddb141
[  806.403629] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.403847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.403979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.404325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.404413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.404677] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.404859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.404978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.405197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.405445] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.405602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.405718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.405932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.406077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.406223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.406471] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.406626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.406753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.406961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.407102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.407336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.407480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.407604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.407813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.407966] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  806.408235] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cb5a9dc0951c474592ed80e30620efa4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  806.414048] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cb5a9dc0951c474592ed80e30620efa4
[  806.442682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  806.443793] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  806.443793] env[60890]: ERROR oslo_messaging.rpc.server 
[  806.520995] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.520995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.521170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.521375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.521531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.521791] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.521951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.522072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.522290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.522933] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.523115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.523238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.523458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.523623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.523782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.524056] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.524216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.524341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.524550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.524688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.524967] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.525132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.525253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.525466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.525604] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  806.525838] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 458608a34d6048d1996ed586ae95d1b8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  806.531314] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 458608a34d6048d1996ed586ae95d1b8
[  806.910054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  806.910597] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  806.910597] env[60890]: ERROR oslo_messaging.rpc.server 
[  806.989165] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.989438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.989875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.990170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.990389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.990588] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.990782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.990915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.991132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.991380] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.991540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.991661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.991874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.992035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.992193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.992445] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.992602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.992811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.992961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.993102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  806.993342] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  806.993497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  806.993617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  806.993824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  806.993964] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  806.994199] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8a6738c054c64f9a8b5e21f2bd0904e7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  807.000772] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8a6738c054c64f9a8b5e21f2bd0904e7
[  807.028042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  807.028042] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  807.028042] env[60890]: ERROR oslo_messaging.rpc.server 
[  807.496874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  807.497444] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  807.497444] env[60890]: ERROR oslo_messaging.rpc.server 
[  807.530140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.530334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.530457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.530721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.530863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.531116] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.531267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.531382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.531591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.531830] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.531983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.532118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.532327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.532466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.532613] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.532918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.533072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.533189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.533393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.533520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.533756] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.533903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.534018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.534219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.534350] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  807.534579] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 72bbc33572c649b3b2d2acb0a9a21742 in queue reply_156db6c4256a4c58884afce15e6f5340
[  807.540980] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 72bbc33572c649b3b2d2acb0a9a21742
[  807.600782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.600998] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.601126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.601472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.601633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.601897] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.602059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.602178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.602399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.602647] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.602810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.602925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.603133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.603274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.603422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.603746] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.603906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.604040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.604254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.604390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  807.604627] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  807.604832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  807.604960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  807.605171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  807.605309] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  807.605542] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a8a2d5452b3a4ee9aa75aa43ffccbd85 in queue reply_156db6c4256a4c58884afce15e6f5340
[  807.611561] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a8a2d5452b3a4ee9aa75aa43ffccbd85
[  808.035521] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.035869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.035869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.036151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.036300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.036560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.036718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.036836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.037104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.037327] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.037482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.037598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.037805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.037949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.038094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.038342] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.038497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.038618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.038835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.038955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.039189] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.039343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.039459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.039665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.039799] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  808.040304] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e097cb58ab1b4d8e8d0eaea3c95e2760 in queue reply_156db6c4256a4c58884afce15e6f5340
[  808.041976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  808.042410] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  808.042410] env[60890]: ERROR oslo_messaging.rpc.server 
[  808.045399] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e097cb58ab1b4d8e8d0eaea3c95e2760
[  808.107831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  808.108354] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  808.108354] env[60890]: ERROR oslo_messaging.rpc.server 
[  808.542960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  808.543495] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  808.543495] env[60890]: ERROR oslo_messaging.rpc.server 
[  808.547643] env[60890]: INFO networking_nsxv3.common.synchronization [req-89a8da7e-34c3-4eae-b0bd-603529deaa73 req-77074304-4c6f-45aa-8ea0-5b39ea3a4f64 tempest-ImagesTestJSON-598093153 tempest-ImagesTestJSON-598093153-project-member] Enqueued Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  808.547822] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  808.548388] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.548519] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.548587] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.548779] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.548878] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.550046] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f0a2229c-87aa-4f25-9157-b1b2e986b23e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.550046] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.550046] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.550046] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.550046] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.550046] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.550046] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.550046] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.550046] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.550046] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f0a2229c-87aa-4f25-9157-b1b2e986b23e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  808.550046] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e8e2a0176dd740c6b3e00c11cb16aa21 in queue reply_156db6c4256a4c58884afce15e6f5340
[  808.565035] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e8e2a0176dd740c6b3e00c11cb16aa21
[  808.704996] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.705212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.705343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.705619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.705775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.712104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.712104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.712104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.712104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.712104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.712104] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  808.712104] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 18ab3a9c276d4ef6a599644cb2060a18 in queue reply_156db6c4256a4c58884afce15e6f5340
[  808.714651] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 18ab3a9c276d4ef6a599644cb2060a18
[  808.823701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.823917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.824073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.824352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.824501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.824825] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.824953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.825077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.825297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.825544] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.826376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.826376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.826376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.826376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.826376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.826618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.826727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.826848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.827102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.827193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.827432] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.827588] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.827706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.827911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.828069] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  808.828358] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6de3cde9a1bd4f7b9f94fdb2c89e0cdb in queue reply_156db6c4256a4c58884afce15e6f5340
[  808.832959] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6de3cde9a1bd4f7b9f94fdb2c89e0cdb
[  808.882077] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.882077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.882227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.882780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.882780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.882914] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.883640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.883640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.883640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.883640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.883916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.883916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.884094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.884235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.884382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.884627] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.884797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.884926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.885135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.885266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  808.885503] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  808.885654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  808.885786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  808.886070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  808.886207] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  808.886440] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fd4d483f83834c88ab85cae621968402 in queue reply_156db6c4256a4c58884afce15e6f5340
[  808.891093] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fd4d483f83834c88ab85cae621968402
[  809.055429] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.055790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.055790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.055969] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.056044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.056209] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.056313] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.056377] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.056530] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.056608] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.056695] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e already deleted.
[  809.056777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.088681] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.088872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.089001] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.089247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.089389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.089636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-645 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.089981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.090208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.090439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.090689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.090846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.090969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.091178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.091318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.091464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-645" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.091712] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.091866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.092020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.092559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.092726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.093047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d809d46-1d0a-4224-91b9-afab8567859f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.093206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.093326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.093535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.093670] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d809d46-1d0a-4224-91b9-afab8567859f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  809.093926] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 97e57cb1645a41dda300007a6cbbd8c6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  809.099633] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 97e57cb1645a41dda300007a6cbbd8c6
[  809.211481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  809.212070] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  809.212070] env[60890]: ERROR oslo_messaging.rpc.server 
[  809.330016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  809.330540] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  809.330540] env[60890]: ERROR oslo_messaging.rpc.server 
[  809.388576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  809.389019] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  809.389019] env[60890]: ERROR oslo_messaging.rpc.server 
[  809.596754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d809d46-1d0a-4224-91b9-afab8567859f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  809.597939] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  809.597939] env[60890]: ERROR oslo_messaging.rpc.server 
[  809.760943] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.761161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.761291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.761566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.761708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.761970] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.762123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.762240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.762454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.762693] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.762843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.762959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.763335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.763537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.763733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.764045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.764247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.764402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.764654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.764848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.765140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.765342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.765497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.765911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.765911] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  809.766205] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0ff61276556c4c6292536c832b31b413 in queue reply_156db6c4256a4c58884afce15e6f5340
[  809.772546] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0ff61276556c4c6292536c832b31b413
[  809.898039] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.898039] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.898039] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.898039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.898597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.898597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.898597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.898695] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.898795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.898913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.899121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.899255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.899494] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.899649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.899853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.900124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.900268] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  809.900499] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 52f8d46a394f441a8b47601db634ebf2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  809.906917] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 52f8d46a394f441a8b47601db634ebf2
[  809.929412] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.929622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.929751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.930022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.930296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.930576] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.930741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.931001] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.931291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.931618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.931810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.932055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.932298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.932750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.933004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.933306] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.933479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.933603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.933858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.933948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  809.934690] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  809.934690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  809.934844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  809.935031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  809.935174] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  809.935410] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f434552910fb4b96819a63cb7bc52b4e in queue reply_156db6c4256a4c58884afce15e6f5340
[  809.940636] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f434552910fb4b96819a63cb7bc52b4e
[  810.268345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  810.268916] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  810.268916] env[60890]: ERROR oslo_messaging.rpc.server 
[  810.355665] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.356344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.356344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.356482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.361701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.361701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.361701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.361701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.361701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.361701] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  810.361701] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg df0904a7abf44f66a9d0a56f74a7c87b in queue reply_156db6c4256a4c58884afce15e6f5340
[  810.365912] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg df0904a7abf44f66a9d0a56f74a7c87b
[  810.402005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  810.402481] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  810.402481] env[60890]: ERROR oslo_messaging.rpc.server 
[  810.404074] env[60890]: INFO networking_nsxv3.common.synchronization [req-b111699a-7ed5-4eb8-93b3-afaf56ab7f0b req-09caee27-28ce-46c9-a046-da531bfec25b tempest-ServerActionsV293TestJSON-1045119592 tempest-ServerActionsV293TestJSON-1045119592-project-member] Enqueued Resource ID: e717cb10-dc75-40d9-a823-032d7367566c with Priority: HIGHEST for action security_group_members
[  810.404735] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e717cb10-dc75-40d9-a823-032d7367566c with Priority: HIGHEST for action security_group_members
[  810.405074] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.405201] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.405269] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.405456] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.405537] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.405684] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e717cb10-dc75-40d9-a823-032d7367566c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.405779] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.405842] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.405988] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.406124] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.406215] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.406273] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.406407] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.406484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.406570] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e717cb10-dc75-40d9-a823-032d7367566c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  810.406740] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 66d81f7aff7e4bc8a4d0745a78ff75cc in queue reply_156db6c4256a4c58884afce15e6f5340
[  810.414563] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 66d81f7aff7e4bc8a4d0745a78ff75cc
[  810.437504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  810.437970] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  810.437970] env[60890]: ERROR oslo_messaging.rpc.server 
[  810.864358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  810.864358] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  810.864358] env[60890]: ERROR oslo_messaging.rpc.server 
[  810.896588] env[60890]: INFO networking_nsxv3.common.synchronization [req-ab32a938-d056-49a8-9a0e-0ab892c4d537 req-313e70f9-b8b6-4cd1-a15d-f00f999ba77c tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 with Priority: HIGHEST for action security_group_members
[  810.896588] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 with Priority: HIGHEST for action security_group_members
[  810.896588] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.896588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.896588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.896588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.896588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.896588] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.896588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.896588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.896993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.896993] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.896993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.896993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.898111] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.898111] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.898111] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f6f508fa-9e0c-4458-b197-ea1aab3b5c03',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  810.898111] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 649a349e4a1041838879885040f68fa4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  810.906939] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 649a349e4a1041838879885040f68fa4
[  810.908496] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.908672] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.908786] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.909043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.909225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.909469] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.909627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.909734] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.909936] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.910068] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.910199] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e717cb10-dc75-40d9-a823-032d7367566c already deleted.
[  810.911398] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e717cb10-dc75-40d9-a823-032d7367566c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.986726] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.987039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.987219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.987547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.987752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.988180] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.988404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.988739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.989156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.989478] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.989711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.989884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.990143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.990342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.990541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.990845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.991071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.991240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.991497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.991685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  810.991984] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  810.992210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  810.992377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  810.992701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  810.992977] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  810.993269] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 74ce71ffd48342e59aaf5902f0594151 in queue reply_156db6c4256a4c58884afce15e6f5340
[  810.998944] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 74ce71ffd48342e59aaf5902f0594151
[  811.010731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.010731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.010731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.010731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.010731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.010731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.010731] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  811.010731] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7b8e46cd926c41aea01f4d35a585431e in queue reply_156db6c4256a4c58884afce15e6f5340
[  811.016820] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7b8e46cd926c41aea01f4d35a585431e
[  811.063875] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.064084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.064214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.064474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.064612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.064895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.065051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.065167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.065378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.065617] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.065787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.065904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.066104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.066240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.066385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.066627] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.066775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.066889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.067091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.067220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.067449] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.067598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.067713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.067928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.068074] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  811.068301] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b7172f86a4e9431a884d10130cad7903 in queue reply_156db6c4256a4c58884afce15e6f5340
[  811.081630] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b7172f86a4e9431a884d10130cad7903
[  811.236850] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.237059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.237188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.237460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.237607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.237865] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.238025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.238145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.238366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.238608] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.238764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.238885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.239093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.239308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.239465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.239714] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.239895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.240045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.240262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.240401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.240749] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.240923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.241046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.241312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.241451] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  811.241674] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 50aac4f37dab45fb93debf2c4ca20aac in queue reply_156db6c4256a4c58884afce15e6f5340
[  811.246309] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 50aac4f37dab45fb93debf2c4ca20aac
[  811.401184] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.401518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.401518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.401630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.401719] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.401913] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  811.402015] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  811.402083] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  811.402228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  811.402304] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.402387] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 already deleted.
[  811.402469] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.495621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  811.496498] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  811.496498] env[60890]: ERROR oslo_messaging.rpc.server 
[  811.513667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  811.513775] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  811.513775] env[60890]: ERROR oslo_messaging.rpc.server 
[  811.573675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  811.573675] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  811.573675] env[60890]: ERROR oslo_messaging.rpc.server 
[  811.743963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  811.743963] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  811.743963] env[60890]: ERROR oslo_messaging.rpc.server 
[  812.066032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.066032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.066032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.066488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.066488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.066488] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-796 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.066747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.066747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.066968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.067272] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.067400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.067523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.067751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.067876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.068055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-796" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.068310] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.068470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.068591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.068804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.068945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.069185] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-29a652af-92bc-4514-bba0-ad2bc2778f48 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.069340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.069459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.069668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.069846] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('29a652af-92bc-4514-bba0-ad2bc2778f48',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  812.070130] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 561bcdbc67df4548a58ccfaf6b703988 in queue reply_156db6c4256a4c58884afce15e6f5340
[  812.083369] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 561bcdbc67df4548a58ccfaf6b703988
[  812.120085] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.120306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.120438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.120711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.120861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.121427] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.121617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.121742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.122005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.122272] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.122611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.122611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.122828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.122984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.123141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.123398] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.123557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.123681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.123952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.124135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.124387] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.124553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.124729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.124968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.125132] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  812.125371] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0ed628df06114d8884db04f9f9ddd60e in queue reply_156db6c4256a4c58884afce15e6f5340
[  812.135417] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0ed628df06114d8884db04f9f9ddd60e
[  812.136358] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.136540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.136669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.137241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.137414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.137676] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.137854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.138000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.138224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.138474] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.138635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.138757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.138971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.139118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.139270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.139517] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.139677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.139797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.140024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.140172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.140414] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.140572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.140693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.140935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.141079] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  812.141306] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1695ae8b7d934e2b88ca738347cae2c7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  812.156755] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1695ae8b7d934e2b88ca738347cae2c7
[  812.283750] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.283957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.284107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.284374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.284523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.284821] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.284986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.285109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.285332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.285581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.285738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.285887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.286081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.286223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.286374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.286625] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.286783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.286906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.287122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.287255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.287493] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  812.287650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  812.287770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  812.287977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  812.288130] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  812.288364] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg be84ff2eba074cc8a0be1023244b1d92 in queue reply_156db6c4256a4c58884afce15e6f5340
[  812.292805] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg be84ff2eba074cc8a0be1023244b1d92
[  812.572750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-29a652af-92bc-4514-bba0-ad2bc2778f48" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  812.573272] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  812.573272] env[60890]: ERROR oslo_messaging.rpc.server 
[  812.635265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  812.635802] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  812.635802] env[60890]: ERROR oslo_messaging.rpc.server 
[  812.643535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  812.643946] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  812.643946] env[60890]: ERROR oslo_messaging.rpc.server 
[  812.791042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  812.791591] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  812.791591] env[60890]: ERROR oslo_messaging.rpc.server 
[  813.205951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.205951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.205951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.205951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.205951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.208184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.208184] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.210401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.210401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.210401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.210401] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  813.210401] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4f372a22e9a447a8ad7a9ff2c0fb1d27 in queue reply_156db6c4256a4c58884afce15e6f5340
[  813.215099] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4f372a22e9a447a8ad7a9ff2c0fb1d27
[  813.222447] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.222447] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.222447] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.222447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.224580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.224886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.225199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.225594] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.225917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.226293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.226726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.227005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.227460] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.227800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.228092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.228433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.228706] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  813.229064] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9797733085fd4a3bb4be8258304b4299 in queue reply_156db6c4256a4c58884afce15e6f5340
[  813.234915] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9797733085fd4a3bb4be8258304b4299
[  813.611724] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.611724] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.611724] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.611724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.613998] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.614222] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.614397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.614521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.614746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.614924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.615183] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.615341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.615461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.615674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.615836] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  813.616089] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fae81bd165db4f149bb3d274958b2a4b in queue reply_156db6c4256a4c58884afce15e6f5340
[  813.621339] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fae81bd165db4f149bb3d274958b2a4b
[  813.665682] env[60890]: INFO networking_nsxv3.common.synchronization [req-6969668b-1d63-41ce-90ab-0ca814391d4d req-966c656f-62fe-4fc5-8ba1-49567ac74856 tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  813.665682] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  813.665682] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.665682] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.665682] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.666001] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.666001] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.666001] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.666115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.666115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.667016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.667016] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  813.667016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  813.667016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  813.667016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  813.667016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.667016] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  813.667016] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7a5041425dbe4b23b223bcc761d1d1dd in queue reply_156db6c4256a4c58884afce15e6f5340
[  813.675836] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7a5041425dbe4b23b223bcc761d1d1dd
[  813.711012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  813.711488] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  813.711488] env[60890]: ERROR oslo_messaging.rpc.server 
[  813.731356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  813.731810] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  813.731810] env[60890]: ERROR oslo_messaging.rpc.server 
[  814.118249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  814.118676] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  814.118676] env[60890]: ERROR oslo_messaging.rpc.server 
[  814.169121] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.169290] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.169356] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.169581] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.169675] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.169868] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.169970] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.170035] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.170183] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.170260] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.170344] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  814.170444] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.289053] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.289313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.289446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.289711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.289887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.290162] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.291145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.291145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.291145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.291145] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.291145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.291389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.291389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.291602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.292107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.292378] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.292543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.292677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.292929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.293072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.293314] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.293470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.293591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.293800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.293941] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  814.294177] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 671e46a93b1b4c13ba834f40e8237bd3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  814.299830] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 671e46a93b1b4c13ba834f40e8237bd3
[  814.314150] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.314349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.314479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.314745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.314938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.315205] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.315366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.315488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.315711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.315958] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.318659] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.318659] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.318659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.318659] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  814.318659] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d48134c177cc49daa2345b7f21a98141 in queue reply_156db6c4256a4c58884afce15e6f5340
[  814.324205] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d48134c177cc49daa2345b7f21a98141
[  814.657235] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.657235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.657235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.657618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.657738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.658080] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.658304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.658457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.658780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.659018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.659217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.659370] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.659636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.659821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.660016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.660348] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.660549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.660703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.660944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.661095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.661406] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  814.661611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  814.661765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  814.662010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  814.662179] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  814.662475] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3321d5f71bb54f08b5893c3536386f7f in queue reply_156db6c4256a4c58884afce15e6f5340
[  814.667657] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3321d5f71bb54f08b5893c3536386f7f
[  814.796686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  814.797203] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  814.797203] env[60890]: ERROR oslo_messaging.rpc.server 
[  814.821247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  814.821681] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  814.821681] env[60890]: ERROR oslo_messaging.rpc.server 
[  815.131417] env[60890]: INFO networking_nsxv3.common.synchronization [req-6969668b-1d63-41ce-90ab-0ca814391d4d req-c229e84e-164a-4655-906f-93e8be9fe383 tempest-ServersTestMultiNic-704046723 tempest-ServersTestMultiNic-704046723-project-member] Enqueued Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  815.132686] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b190120e-95be-4670-a5a0-36b3547a8bcf with Priority: HIGHEST for action security_group_members
[  815.132686] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.132686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.132686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.132686] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.132877] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.132877] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b190120e-95be-4670-a5a0-36b3547a8bcf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.132997] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.133039] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.133188] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.133323] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.133427] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.133465] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.133604] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.133682] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.133773] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b190120e-95be-4670-a5a0-36b3547a8bcf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  815.133950] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 810b469577dc4e9bbb3999d1d027046d in queue reply_156db6c4256a4c58884afce15e6f5340
[  815.154417] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 810b469577dc4e9bbb3999d1d027046d
[  815.165597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  815.165597] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  815.165597] env[60890]: ERROR oslo_messaging.rpc.server 
[  815.368871] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.369091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.369222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.369594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.369675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.369894] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.370053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.370174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.370394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.370676] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.370843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.370967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.371180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.371326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.371477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.371730] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.371885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.372016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.372236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.372375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.372614] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.372783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.372950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.373169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.373312] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  815.373546] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5447b4a27ac9418abf3f0177fa7f8a07 in queue reply_156db6c4256a4c58884afce15e6f5340
[  815.381764] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5447b4a27ac9418abf3f0177fa7f8a07
[  815.461981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.462172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.462303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.462576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.462718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.463018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-531 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.463181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.463301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.463519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.463768] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.463929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.464065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.464280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.464425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.464575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-531" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.464865] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.465028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.465148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.465357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.465493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.465731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-af9c3695-ede0-4b11-ac58-f71f41cb02ea taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.465892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.466008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.466216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.466354] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('af9c3695-ede0-4b11-ac58-f71f41cb02ea',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  815.466592] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5e0bdb6a4dc646089ff1dc5a00100377 in queue reply_156db6c4256a4c58884afce15e6f5340
[  815.472598] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5e0bdb6a4dc646089ff1dc5a00100377
[  815.636405] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.636567] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.636638] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.636854] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.636950] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.637158] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.637261] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.637324] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.637475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.637555] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.637642] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b190120e-95be-4670-a5a0-36b3547a8bcf already deleted.
[  815.637721] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b190120e-95be-4670-a5a0-36b3547a8bcf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.712246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.712551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.712598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.712838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.712987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.713248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.713408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.713529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.713748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.713995] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.714156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.714279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.714486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.714628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.714794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.715074] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.715234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.715355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.715566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.715705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.715988] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  815.716175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  815.716297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  815.716505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  815.716640] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  815.716873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f6c8fde954544d06ace201f38721eae2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  815.723998] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f6c8fde954544d06ace201f38721eae2
[  815.875280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  815.875794] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  815.875794] env[60890]: ERROR oslo_messaging.rpc.server 
[  815.968761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-af9c3695-ede0-4b11-ac58-f71f41cb02ea" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  815.969264] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  815.969264] env[60890]: ERROR oslo_messaging.rpc.server 
[  816.218856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  816.219369] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  816.219369] env[60890]: ERROR oslo_messaging.rpc.server 
[  816.614630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.614843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.614982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.615252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.615398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.615652] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.615816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.615932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.616173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.616418] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.616569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.616683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.616891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.617030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.617176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.617421] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.617571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.617689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.617893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.618022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.618260] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.618409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.618525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.618727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.618867] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  816.619095] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 442a06355f554f27b359181d36ae4f99 in queue reply_156db6c4256a4c58884afce15e6f5340
[  816.624374] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 442a06355f554f27b359181d36ae4f99
[  816.961188] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.961458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.961542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.961790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.961939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.962201] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.962359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.962477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.962712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.962966] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.963125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.963246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.963455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.963597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.963749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.964011] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.964202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.964323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.964533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.964669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  816.965103] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  816.965313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  816.965451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  816.965685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  816.965879] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  816.966146] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6152d707862c4ad3b9b1d15e387a6ee7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  816.972440] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6152d707862c4ad3b9b1d15e387a6ee7
[  817.122833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  817.122833] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  817.122833] env[60890]: ERROR oslo_messaging.rpc.server 
[  817.199064] env[60890]: INFO networking_nsxv3.common.synchronization [req-768981e8-8509-4f96-9e5e-46d73f09920b req-4a4d3a9a-ea01-4874-8f92-5bba048bb92f tempest-SecurityGroupsTestJSON-806588483 tempest-SecurityGroupsTestJSON-806588483-project-member] Enqueued Resource ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 with Priority: HIGHEST for action security_group_members
[  817.199064] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 with Priority: HIGHEST for action security_group_members
[  817.199449] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.199449] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.199449] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.199608] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.199698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.199845] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.199940] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.200037] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.200172] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.200305] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.200391] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.200448] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.200582] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.200658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.200751] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('384e1ef2-f71b-4a37-9e25-4fafe57f50e8',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  817.200926] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 68adf2258ce740079b5d1e60648a06a4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  817.212766] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 68adf2258ce740079b5d1e60648a06a4
[  817.256093] env[60890]: INFO networking_nsxv3.common.synchronization [req-3acebeb8-f188-4a3b-ac20-71acd5e5d457 req-15ae2d27-b3ed-4427-9049-c2af7fcc3db4 tempest-AttachVolumeTestJSON-389769224 tempest-AttachVolumeTestJSON-389769224-project-member] Enqueued Resource ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 with Priority: HIGHEST for action security_group_members
[  817.256093] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 with Priority: HIGHEST for action security_group_members
[  817.256093] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.256093] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-2697824d-b6cb-4c8c-8cbe-cd77309bea77 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.256093] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.256093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.256093] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('2697824d-b6cb-4c8c-8cbe-cd77309bea77',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  817.256093] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b887c08adfa54920ada5dab7247a6035 in queue reply_156db6c4256a4c58884afce15e6f5340
[  817.265995] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b887c08adfa54920ada5dab7247a6035
[  817.467929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  817.468508] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  817.468508] env[60890]: ERROR oslo_messaging.rpc.server 
[  817.708497] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.708659] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.708735] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.709033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.709142] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.709344] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.709449] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.709512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.709672] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.709751] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.709838] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 384e1ef2-f71b-4a37-9e25-4fafe57f50e8 already deleted.
[  817.709917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-384e1ef2-f71b-4a37-9e25-4fafe57f50e8" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.758544] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.758710] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.758757] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.758978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.759075] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.759275] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.759368] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.759435] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.759588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.759695] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.759753] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 already deleted.
[  817.759835] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.907209] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.907431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.907560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.907838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.907992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.908283] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.908441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.908560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.908783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.909031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.909190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.909311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.909524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.909667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.909819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.910082] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.910303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.910435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.910650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.910790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  817.911034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  817.911194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  817.911314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  817.912158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  817.912313] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  817.912557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 497a9a639f87438b83fd96522deb9cd6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  817.918186] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 497a9a639f87438b83fd96522deb9cd6
[  818.042233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.042516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.042561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.042874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.043016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.043298] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.043464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.043586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.043805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.044346] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.044552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.044682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.044954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.045302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.045476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.045732] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.045916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.046042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.046258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.046397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.046636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.046793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.046915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.047135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.047265] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  818.047495] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b9dad11968274c75a92e0f84bceb8b12 in queue reply_156db6c4256a4c58884afce15e6f5340
[  818.053768] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b9dad11968274c75a92e0f84bceb8b12
[  818.415249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  818.415775] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  818.415775] env[60890]: ERROR oslo_messaging.rpc.server 
[  818.516036] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.516278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.516417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.516685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.516835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.517093] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.517254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.517376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.517667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.517939] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.518099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.518220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.518430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.518572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.518722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.518985] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.519171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.519294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.519505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.519642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.519945] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.520063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.520195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.520437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.520588] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f28f29ec-99a0-48bf-b2c9-03ae2ce6824e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  818.520824] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5c5d24f180dc47bbb7b67eb23b5c3fd6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  818.529923] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5c5d24f180dc47bbb7b67eb23b5c3fd6
[  818.550627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  818.551151] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  818.551151] env[60890]: ERROR oslo_messaging.rpc.server 
[  818.869860] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.870023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.870149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.870423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.870565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.870823] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.870987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.871118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.871296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.871537] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.871689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.871863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.872109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.872284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.872393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.872645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.872815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.872941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.873150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.873281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.877372] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.877372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.877372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.877372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.877372] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  818.877372] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg db179d41672e4eaf99beea00871d72ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  818.877920] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.877920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.877920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.878045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.878116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.878355] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.878506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.878621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.878832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.879075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.879228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.879344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.879675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.879763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.879830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.880100] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.880257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.880375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.880583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.880713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  818.880946] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  818.881098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  818.881217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  818.881419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  818.881546] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  818.881764] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8c3b7669bed34174a8029cfd1e5285f0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  818.987972] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg db179d41672e4eaf99beea00871d72ad
[  818.990322] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8c3b7669bed34174a8029cfd1e5285f0
[  819.022703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f28f29ec-99a0-48bf-b2c9-03ae2ce6824e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  819.023161] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  819.023161] env[60890]: ERROR oslo_messaging.rpc.server 
[  819.376231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  819.376752] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  819.376752] env[60890]: ERROR oslo_messaging.rpc.server 
[  819.383581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  819.384009] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  819.384009] env[60890]: ERROR oslo_messaging.rpc.server 
[  819.793305] env[60890]: INFO networking_nsxv3.common.synchronization [req-89a8da7e-34c3-4eae-b0bd-603529deaa73 req-ee701fe9-b3e8-4fd4-8859-cc9d3fd5f3be tempest-ImagesTestJSON-598093153 tempest-ImagesTestJSON-598093153-project-member] Enqueued Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  819.793843] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  819.794214] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.794283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.794350] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.794525] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.794608] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.794844] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f0a2229c-87aa-4f25-9157-b1b2e986b23e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.794928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.794993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.795183] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.795328] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.795434] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.795502] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.795719] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.795819] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.795925] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f0a2229c-87aa-4f25-9157-b1b2e986b23e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  819.796131] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6dce8aed95584b5896acfb02a2dba298 in queue reply_156db6c4256a4c58884afce15e6f5340
[  819.803747] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6dce8aed95584b5896acfb02a2dba298
[  819.943997] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.944267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.944405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.944687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.945202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.945511] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.945678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.945833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.946234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.946527] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.946694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.946821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.947611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.948045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.948265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.949035] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.949450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.949608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.949936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.950155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.950550] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.950753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.950903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.951234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.951462] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  819.951817] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4874be7429aa45938bb10e4e6d8f4be9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  819.957286] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4874be7429aa45938bb10e4e6d8f4be9
[  819.958589] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.958774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.958906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.959177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.959321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.959600] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.959767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.959878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.960131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.960660] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.960830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.960957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.961176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.961324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.961478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.961844] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.962015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.962139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.962358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.962499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  819.962833] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  819.963004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  819.963143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  819.963336] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  819.963474] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  819.963703] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b6eb6b9e73e642df85951e54cf2d5c05 in queue reply_156db6c4256a4c58884afce15e6f5340
[  819.968213] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b6eb6b9e73e642df85951e54cf2d5c05
[  820.298253] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  820.298390] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  820.298448] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  820.298669] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  820.298770] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  820.298962] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  820.299065] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  820.299127] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  820.299279] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  820.299359] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  820.299446] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e already deleted.
[  820.299523] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  820.453821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  820.454353] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  820.454353] env[60890]: ERROR oslo_messaging.rpc.server 
[  820.465827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  820.466453] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  820.466453] env[60890]: ERROR oslo_messaging.rpc.server 
[  821.086230] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.086230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.086230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.086230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.086230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.086230] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.086230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.086230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.086230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.086798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.086798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.086798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.086798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.086950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.087734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.087734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.087734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.087734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.087925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.087925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.088144] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.088482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.088482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.088726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.088765] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  821.091291] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 267d23de99a14d0b865d8a0ff21720cb in queue reply_156db6c4256a4c58884afce15e6f5340
[  821.093935] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.094115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.094240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.094467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.094605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.094878] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.095045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.095166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.095378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.095623] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.095779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.095920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.096191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.096298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.096451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.096701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.096857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.096980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.097193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.097327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.097568] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-64ef44aa-9a56-4442-bd6f-005c143252a4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  821.097722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  821.097846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  821.098058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  821.098194] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('64ef44aa-9a56-4442-bd6f-005c143252a4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  821.098426] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9d72ccac561043e7a6375dafe7e44451 in queue reply_156db6c4256a4c58884afce15e6f5340
[  821.099310] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 267d23de99a14d0b865d8a0ff21720cb
[  821.104879] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9d72ccac561043e7a6375dafe7e44451
[  821.591277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  821.591898] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  821.591898] env[60890]: ERROR oslo_messaging.rpc.server 
[  821.601950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-64ef44aa-9a56-4442-bd6f-005c143252a4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  821.602246] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  821.602246] env[60890]: ERROR oslo_messaging.rpc.server 
[  822.282577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.282800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.282965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.283200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.283344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.283596] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.283749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.283868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.284110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.284358] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.284587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.284675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.285020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.285100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.286255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.286255] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.286255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.286255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.286255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.286255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.286602] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.286602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.286602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.286938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.286938] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  822.290487] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4f07f1bdcb9243c4a2ec12d8693a53d4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  822.308382] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4f07f1bdcb9243c4a2ec12d8693a53d4
[  822.730725] env[60890]: INFO networking_nsxv3.common.synchronization [req-ab32a938-d056-49a8-9a0e-0ab892c4d537 req-13409470-5c46-40ff-89e8-3b50773301b4 tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 with Priority: HIGHEST for action security_group_members
[  822.731292] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 with Priority: HIGHEST for action security_group_members
[  822.731632] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.731789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.731848] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.732133] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.732238] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.732587] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.732587] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.732702] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.732810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.732961] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.733055] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.733117] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.733342] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.733429] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.733527] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f6f508fa-9e0c-4458-b197-ea1aab3b5c03',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  822.733845] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 46982afac61d456793ad144c9d1c83b3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  822.741142] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 46982afac61d456793ad144c9d1c83b3
[  822.798316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  822.798316] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  822.798316] env[60890]: ERROR oslo_messaging.rpc.server 
[  822.898684] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.898684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.898684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.898684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.898854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.899189] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.899285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.899324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.899538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.899782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.899940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.900074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.900289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.900446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.900572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.915611] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.915611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.915611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.915611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.915611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  822.915611] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  822.915611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  822.915611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  822.915611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  822.915611] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  822.915611] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg da0739133efa4f22ba5358a93d279fd1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  822.915611] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg da0739133efa4f22ba5358a93d279fd1
[  823.235588] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  823.235810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  823.235901] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  823.236155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  823.236247] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  823.236440] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  823.236529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  823.236589] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  823.236736] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  823.236808] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  823.236900] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f6f508fa-9e0c-4458-b197-ea1aab3b5c03 already deleted.
[  823.237001] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f6f508fa-9e0c-4458-b197-ea1aab3b5c03" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  823.406345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  823.406866] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  823.406866] env[60890]: ERROR oslo_messaging.rpc.server 
[  823.943929] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  823.944216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  823.944546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  823.944626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  823.944739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  823.945697] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  823.945697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  823.945697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  823.945697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  823.945966] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  823.945966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  823.946062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  823.946207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  823.946347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  823.946500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  823.946758] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  823.946918] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  823.947623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  823.947623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  823.947623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  823.947623] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  823.947850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  823.947892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  823.950190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  823.950888] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  823.951175] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 544bd6cd7f824de2bef28b0e59cd9abb in queue reply_156db6c4256a4c58884afce15e6f5340
[  823.960540] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 544bd6cd7f824de2bef28b0e59cd9abb
[  824.457636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  824.458161] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  824.458161] env[60890]: ERROR oslo_messaging.rpc.server 
[  824.666917] env[60890]: INFO networking_nsxv3.common.synchronization [None req-cb7aa21f-ed82-4d5b-a63b-c5cae5729ec7 None None] Enqueued Resource ID: 6bd73583-7fc6-4f6f-b716-089b997a9b89 with Priority: HIGHEST for action port
[  824.666917] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6bd73583-7fc6-4f6f-b716-089b997a9b89 with Priority: HIGHEST for action port
[  824.666917] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  824.666917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  824.666917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  824.666917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  824.666917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  824.666917] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-6bd73583-7fc6-4f6f-b716-089b997a9b89 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  824.666917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-6bd73583-7fc6-4f6f-b716-089b997a9b89" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  824.666917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-6bd73583-7fc6-4f6f-b716-089b997a9b89" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  824.666917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-6bd73583-7fc6-4f6f-b716-089b997a9b89" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  824.666917] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6bd73583-7fc6-4f6f-b716-089b997a9b89',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  824.666917] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 032c69d06a084bc78e29f0870167ca2d in queue reply_156db6c4256a4c58884afce15e6f5340
[  824.792522] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 032c69d06a084bc78e29f0870167ca2d
[  825.085089] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  825.085373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  825.085492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  825.085656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  825.085829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  825.086094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  825.086246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  825.086361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  825.086574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  825.086811] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  825.086963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  825.087077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  825.090571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  825.090752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  825.090929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  825.091228] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  825.091386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  825.091504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  825.091718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  825.091884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  825.092211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  825.092369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  825.092486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  825.092694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  825.092830] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  825.093061] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 275ea196febe4763bb2337c927ef1969 in queue reply_156db6c4256a4c58884afce15e6f5340
[  825.101569] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 275ea196febe4763bb2337c927ef1969
[  825.172475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-6bd73583-7fc6-4f6f-b716-089b997a9b89" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  825.172712] env[60890]: Traceback (most recent call last):
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  825.172763] env[60890]:     timer()
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  825.172763] env[60890]:     cb(*args, **kw)
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  825.172763] env[60890]:     result = function(*args, **kwargs)
[  825.172763] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  825.172763] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  825.172763] env[60890]:     return method(*args, **kwargs)
[  825.172763] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  825.172763] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  825.172763] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  825.172763] env[60890]:     result = self.transport._send(
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  825.172763] env[60890]:     return self._driver.send(target, ctxt, message,
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  825.172763] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  825.172763] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  825.172763] env[60890]:     raise result
[  825.173425] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  825.173425] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  825.595413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  825.595952] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  825.595952] env[60890]: ERROR oslo_messaging.rpc.server 
[  825.786748] env[60890]: INFO networking_nsxv3.common.synchronization [None req-d0556398-f551-414f-b3ae-72c4eae6ee3e None None] Enqueued Resource ID: 35d46f8a-f3d2-4674-8ecd-3e33ca2c5892 with Priority: HIGHEST for action port
[  825.792412] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 35d46f8a-f3d2-4674-8ecd-3e33ca2c5892 with Priority: HIGHEST for action port
[  825.792412] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  825.792412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  825.792412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  825.792412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  825.792412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  825.792412] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-35d46f8a-f3d2-4674-8ecd-3e33ca2c5892 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  825.792412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-35d46f8a-f3d2-4674-8ecd-3e33ca2c5892" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  825.792412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-35d46f8a-f3d2-4674-8ecd-3e33ca2c5892" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  825.792412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-35d46f8a-f3d2-4674-8ecd-3e33ca2c5892" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  825.792412] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('35d46f8a-f3d2-4674-8ecd-3e33ca2c5892',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  825.792412] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 98d1bb47c08943b2a3077a011eb93613 in queue reply_156db6c4256a4c58884afce15e6f5340
[  825.797552] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 98d1bb47c08943b2a3077a011eb93613
[  826.168457] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.168877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.168877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.169050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.169202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.169466] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.169633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.169757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.169984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.170229] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.170386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.170508] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.170722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.170865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.171044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.171302] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.171464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.171587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.171800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.171941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.172203] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.172361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.172483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.172712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.172882] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  826.173090] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 26925db0f6a54ad682ae625fb8efe9a8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  826.233569] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 26925db0f6a54ad682ae625fb8efe9a8
[  826.293420] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-35d46f8a-f3d2-4674-8ecd-3e33ca2c5892" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.293678] env[60890]: Traceback (most recent call last):
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  826.293747] env[60890]:     timer()
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  826.293747] env[60890]:     cb(*args, **kw)
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  826.293747] env[60890]:     result = function(*args, **kwargs)
[  826.293747] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  826.293747] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  826.293747] env[60890]:     return method(*args, **kwargs)
[  826.293747] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  826.293747] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  826.293747] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  826.293747] env[60890]:     result = self.transport._send(
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  826.293747] env[60890]:     return self._driver.send(target, ctxt, message,
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  826.293747] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  826.293747] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  826.293747] env[60890]:     raise result
[  826.293747] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  826.293747] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  826.558906] env[60890]: INFO networking_nsxv3.common.synchronization [None req-a511c5f7-867a-4225-8d94-83ef6df50afb None None] Enqueued Resource ID: 8b2c3778-5833-4a28-9015-8bd0c4c38504 with Priority: HIGHEST for action port
[  826.559217] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8b2c3778-5833-4a28-9015-8bd0c4c38504 with Priority: HIGHEST for action port
[  826.560197] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.560327] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.560400] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.560597] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.560677] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.560840] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-8b2c3778-5833-4a28-9015-8bd0c4c38504 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.560934] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-8b2c3778-5833-4a28-9015-8bd0c4c38504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.560993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-8b2c3778-5833-4a28-9015-8bd0c4c38504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.561210] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-8b2c3778-5833-4a28-9015-8bd0c4c38504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.561284] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('8b2c3778-5833-4a28-9015-8bd0c4c38504',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  826.561484] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5957823a5f5a47829af304fc18402c17 in queue reply_156db6c4256a4c58884afce15e6f5340
[  826.675027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  826.675628] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  826.675628] env[60890]: ERROR oslo_messaging.rpc.server 
[  826.713167] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5957823a5f5a47829af304fc18402c17
[  826.721697] env[60890]: INFO networking_nsxv3.common.synchronization [None req-f9e544d8-0c5f-4994-be93-29db69977ac9 tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb with Priority: HIGHEST for action security_group_rules
[  826.722180] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb with Priority: HIGHEST for action security_group_rules
[  826.722493] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.722611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.722676] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.722898] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.722980] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.723133] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.723224] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.723276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.723474] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.723546] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('beabdba8-4b41-4b8f-a694-d75a0cc077bb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  826.723748] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 12b78a08288d411488b461717ecb46f7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  826.809172] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 12b78a08288d411488b461717ecb46f7
[  826.837559] env[60890]: INFO networking_nsxv3.common.synchronization [req-c054220d-8cf4-45c3-9d89-8b4cac76bbc1 req-b5bd0c65-7ebb-418e-aab1-a49bf2630f8f tempest-ServerActionsTestJSON-983883571 tempest-ServerActionsTestJSON-983883571-project-member] Enqueued Resource ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 with Priority: HIGHEST for action security_group_members
[  826.838143] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 with Priority: HIGHEST for action security_group_members
[  826.838403] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.838526] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.838593] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.838789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.838881] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.839055] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-68938528-e0ea-4599-af8e-22dc66ee1e55 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.839141] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.839254] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.839351] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.839485] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.839577] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.839640] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.839814] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.839848] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.839921] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('68938528-e0ea-4599-af8e-22dc66ee1e55',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  826.840142] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 73a3abf15874486a8dce1df344435c8b in queue reply_156db6c4256a4c58884afce15e6f5340
[  826.858134] env[60890]: INFO networking_nsxv3.common.synchronization [None req-a7f2682b-40a6-43c4-9218-82090178ebb6 tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb with Priority: HIGHEST for action security_group_rules
[  826.858622] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb with Priority: HIGHEST for action security_group_rules
[  826.858946] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.859075] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.859139] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  826.859334] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  826.859419] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  826.859574] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  826.859654] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  826.860515] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 73a3abf15874486a8dce1df344435c8b
[  827.064642] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-8b2c3778-5833-4a28-9015-8bd0c4c38504" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.064918] env[60890]: Traceback (most recent call last):
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  827.064993] env[60890]:     timer()
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  827.064993] env[60890]:     cb(*args, **kw)
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  827.064993] env[60890]:     result = function(*args, **kwargs)
[  827.064993] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  827.064993] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  827.064993] env[60890]:     return method(*args, **kwargs)
[  827.064993] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  827.064993] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  827.064993] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  827.064993] env[60890]:     result = self.transport._send(
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  827.064993] env[60890]:     return self._driver.send(target, ctxt, message,
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  827.064993] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  827.064993] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  827.064993] env[60890]:     raise result
[  827.064993] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  827.064993] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  827.249151] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.249513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.249513] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.249618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.250673] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.250673] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb already deleted.
[  827.250673] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.250673] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.250673] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.250673] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('beabdba8-4b41-4b8f-a694-d75a0cc077bb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  827.250673] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2c921e75be01412a9687c95c78e77198 in queue reply_156db6c4256a4c58884afce15e6f5340
[  827.269311] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2c921e75be01412a9687c95c78e77198
[  827.284795] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.285011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.285140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.285415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.285558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.285815] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.285965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.286078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.286291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.286538] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.286671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.286785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.287001] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.287160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.287318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.287609] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.287778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.287917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.288483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.288674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.288904] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.289066] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.289187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.289405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.289544] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  827.289779] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9ca0249770c64bd5b557244dc9d8cb87 in queue reply_156db6c4256a4c58884afce15e6f5340
[  827.297028] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9ca0249770c64bd5b557244dc9d8cb87
[  827.342549] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.342707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.342789] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.343010] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.343168] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.343299] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.343402] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.343460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.343615] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.343692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.343785] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 already deleted.
[  827.343948] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.426648] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.426860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.426986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.427275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.427421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.427689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.427848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.427950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.428291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.428544] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.428699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.428816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.429025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.429161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.429306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.429561] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.429710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.430026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.430096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.430171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.430407] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.430557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.430674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.430919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.431056] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  827.431292] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5c520ae69696480495a99534b6baa583 in queue reply_156db6c4256a4c58884afce15e6f5340
[  827.442184] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5c520ae69696480495a99534b6baa583
[  827.772857] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  827.772961] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  827.773044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  827.773225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  827.773325] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.773424] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb already deleted.
[  827.773549] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.792296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  827.792798] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  827.792798] env[60890]: ERROR oslo_messaging.rpc.server 
[  827.940213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  827.940751] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  827.940751] env[60890]: ERROR oslo_messaging.rpc.server 
[  828.416772] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.416772] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-107 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.416772] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.416772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.418215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-107" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.418215] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.418582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.418849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.419295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.419665] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.420103] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-777315b5-4ab4-475b-89c0-467aea3033d0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.420443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.420715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.421052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.421326] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('777315b5-4ab4-475b-89c0-467aea3033d0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  828.421737] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 342116ae1310447b8941b84b33af8338 in queue reply_156db6c4256a4c58884afce15e6f5340
[  828.430616] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 342116ae1310447b8941b84b33af8338
[  828.528312] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.528312] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.528312] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.528312] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.528312] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.528312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.528312] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  828.528312] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b2b25da3c409469784bb0e066bdf813c in queue reply_156db6c4256a4c58884afce15e6f5340
[  828.535785] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b2b25da3c409469784bb0e066bdf813c
[  828.650063] env[60890]: INFO networking_nsxv3.common.synchronization [req-23fe4620-b919-4e16-9f81-4a57d6a3c953 req-74efc97c-46ac-4392-86a9-f48f925b9515 tempest-MultipleCreateTestJSON-1912904963 tempest-MultipleCreateTestJSON-1912904963-project-member] Enqueued Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  828.650063] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  828.650063] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.650063] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-08b77cbe-1697-46da-85d2-54fa8e8cc834 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.650063] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.650063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.650063] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('08b77cbe-1697-46da-85d2-54fa8e8cc834',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  828.650063] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9c93452e13fa404388a9533af1e9cfcb in queue reply_156db6c4256a4c58884afce15e6f5340
[  828.660685] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9c93452e13fa404388a9533af1e9cfcb
[  828.802270] env[60890]: INFO networking_nsxv3.common.synchronization [None req-aed2bd4b-4c88-47f2-ae56-19b9811094d8 None None] Enqueued Resource ID: 1fb2b077-16f1-4b21-aedb-11bc026dcf04 with Priority: HIGHEST for action port
[  828.802270] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 1fb2b077-16f1-4b21-aedb-11bc026dcf04 with Priority: HIGHEST for action port
[  828.802270] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.802270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.802270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.802270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.802270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.802270] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-1fb2b077-16f1-4b21-aedb-11bc026dcf04 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  828.802270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-1fb2b077-16f1-4b21-aedb-11bc026dcf04" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  828.802270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-1fb2b077-16f1-4b21-aedb-11bc026dcf04" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  828.802270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-1fb2b077-16f1-4b21-aedb-11bc026dcf04" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  828.802270] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1fb2b077-16f1-4b21-aedb-11bc026dcf04',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  828.802270] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 05a2f3b2ead040ad89c00ab4619df1ac in queue reply_156db6c4256a4c58884afce15e6f5340
[  828.810266] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 05a2f3b2ead040ad89c00ab4619df1ac
[  828.927843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-777315b5-4ab4-475b-89c0-467aea3033d0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  828.928695] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  828.928695] env[60890]: ERROR oslo_messaging.rpc.server 
[  829.030558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  829.030558] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  829.030558] env[60890]: ERROR oslo_messaging.rpc.server 
[  829.083734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.083734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  829.083734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.083734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.083734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  829.083734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  829.083734] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  829.083734] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 73f2982e9c0f467fb25a9ea9d9e34e54 in queue reply_156db6c4256a4c58884afce15e6f5340
[  829.086106] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 73f2982e9c0f467fb25a9ea9d9e34e54
[  829.086884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  829.087423] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  829.087423] env[60890]: ERROR oslo_messaging.rpc.server 
[  829.150562] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  829.150688] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  829.150760] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  829.150989] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  829.151078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.151416] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  829.151416] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  829.151543] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  829.151616] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  829.151690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.151773] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 already deleted.
[  829.151847] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.306085] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-1fb2b077-16f1-4b21-aedb-11bc026dcf04" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  829.306337] env[60890]: Traceback (most recent call last):
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  829.306385] env[60890]:     timer()
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  829.306385] env[60890]:     cb(*args, **kw)
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  829.306385] env[60890]:     result = function(*args, **kwargs)
[  829.306385] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  829.306385] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  829.306385] env[60890]:     return method(*args, **kwargs)
[  829.306385] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  829.306385] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  829.306385] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  829.306385] env[60890]:     result = self.transport._send(
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  829.306385] env[60890]:     return self._driver.send(target, ctxt, message,
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  829.306385] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  829.306385] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  829.306385] env[60890]:     raise result
[  829.307152] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  829.307152] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  830.059382] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.059699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.059699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.059970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.060174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.060434] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.060774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.060774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.060926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.061174] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.061385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.061534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.061751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.061893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.062046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.062296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.062451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.062678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.062887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.063033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.063277] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.063433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.063552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.063764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.063896] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  830.064583] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 353f7483564e4a1890351e22d2751383 in queue reply_156db6c4256a4c58884afce15e6f5340
[  830.070122] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 353f7483564e4a1890351e22d2751383
[  830.103150] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.103150] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.103150] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.103150] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.103150] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.103150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.103150] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  830.103150] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fe00f06f591c44d4920b5ee7da1972e8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  830.107498] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fe00f06f591c44d4920b5ee7da1972e8
[  830.496737] env[60890]: INFO networking_nsxv3.common.synchronization [req-3acebeb8-f188-4a3b-ac20-71acd5e5d457 req-60d9ad20-96d3-4c68-8f54-0d1e4fa386c4 tempest-AttachVolumeTestJSON-389769224 tempest-AttachVolumeTestJSON-389769224-project-member] Enqueued Resource ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 with Priority: HIGHEST for action security_group_members
[  830.497291] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 with Priority: HIGHEST for action security_group_members
[  830.497607] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.497730] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.497797] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.498032] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.498142] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.498337] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-2697824d-b6cb-4c8c-8cbe-cd77309bea77 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.498437] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.498437] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.498661] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.498713] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.498798] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.498861] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.498998] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.499077] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.499163] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('2697824d-b6cb-4c8c-8cbe-cd77309bea77',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  830.499344] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 009eaccb27e24d8fb0d4d51f537bed47 in queue reply_156db6c4256a4c58884afce15e6f5340
[  830.507076] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 009eaccb27e24d8fb0d4d51f537bed47
[  830.567266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  830.567794] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  830.567794] env[60890]: ERROR oslo_messaging.rpc.server 
[  830.605270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  830.605795] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  830.605795] env[60890]: ERROR oslo_messaging.rpc.server 
[  830.697793] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.698056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.698234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.698625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.698792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.699164] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.699373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.700056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.700309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.700648] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.700880] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.701056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.701322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.701479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.701638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.701925] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.702136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.702297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.704631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.704954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  830.705342] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  830.705523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  830.705684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  830.705965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  830.706126] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  830.706444] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 45b7d50e247c4e5f94735cd49f8760ea in queue reply_156db6c4256a4c58884afce15e6f5340
[  830.712165] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 45b7d50e247c4e5f94735cd49f8760ea
[  831.000968] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.001182] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.001233] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.001410] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.001504] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.001702] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.001804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.001870] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.002025] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.002105] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.002202] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 2697824d-b6cb-4c8c-8cbe-cd77309bea77 already deleted.
[  831.002284] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-2697824d-b6cb-4c8c-8cbe-cd77309bea77" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.112969] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.113415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.113415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.113754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.113965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.114250] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.114421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.114532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.114872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.115157] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.115336] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.115447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.115669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.115807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.115967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.116246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.116408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.116506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.117347] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.117562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.117863] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.118134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.118261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.118474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.118621] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  831.118956] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 73dfd631eaf7438f82449f50e38eab74 in queue reply_156db6c4256a4c58884afce15e6f5340
[  831.134578] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 73dfd631eaf7438f82449f50e38eab74
[  831.208784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  831.209299] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  831.209299] env[60890]: ERROR oslo_messaging.rpc.server 
[  831.621218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  831.621759] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  831.621759] env[60890]: ERROR oslo_messaging.rpc.server 
[  831.756280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.756492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.756624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.756911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.757069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.757349] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.757510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.757632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.757855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.758108] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.758269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.758394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.758610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.758757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.758908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.759164] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.759323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.759444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.759808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.760348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  831.760629] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  831.760867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  831.760997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  831.761217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  831.761359] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  831.761600] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 986c817ca9ac4cf595d6a53c275cea61 in queue reply_156db6c4256a4c58884afce15e6f5340
[  831.767745] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 986c817ca9ac4cf595d6a53c275cea61
[  832.174632] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.174990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.175038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.175267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.175415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.175681] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.175881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.175961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.176272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.176525] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.176689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.176858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.177092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.177242] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.177396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.177647] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.177804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.177927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.178136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.178273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.178512] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.178669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.178788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.178999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.179136] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  832.179372] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9a94e2937d1749a8bd75aa64a53bc3bc in queue reply_156db6c4256a4c58884afce15e6f5340
[  832.185091] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9a94e2937d1749a8bd75aa64a53bc3bc
[  832.263841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  832.264448] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  832.264448] env[60890]: ERROR oslo_messaging.rpc.server 
[  832.681831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  832.682348] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  832.682348] env[60890]: ERROR oslo_messaging.rpc.server 
[  832.813857] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.813857] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.813857] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.813857] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.813857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  832.814972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  832.815490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  832.815757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  832.816119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  832.816596] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  832.816965] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 808031daecf144f3b555719ea3c4ca59 in queue reply_156db6c4256a4c58884afce15e6f5340
[  832.825273] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 808031daecf144f3b555719ea3c4ca59
[  833.232162] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.232462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.232537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.232836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.233034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.233315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.233480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.233601] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.233830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.234082] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.234239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.234359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.234572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.234712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.234934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.235271] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.235451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.235598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.235958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.236171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.236431] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.236595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.236763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.237012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.237177] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  833.237432] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1de8f4d1a4c642b98a4d6dd763398696 in queue reply_156db6c4256a4c58884afce15e6f5340
[  833.248109] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1de8f4d1a4c642b98a4d6dd763398696
[  833.321861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  833.321861] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  833.321861] env[60890]: ERROR oslo_messaging.rpc.server 
[  833.652792] env[60890]: INFO networking_nsxv3.common.synchronization [req-23fe4620-b919-4e16-9f81-4a57d6a3c953 req-96db7445-a132-4f5e-a73c-6a5bf10aa61d tempest-MultipleCreateTestJSON-1912904963 tempest-MultipleCreateTestJSON-1912904963-project-member] Enqueued Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  833.653494] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  833.653737] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.653853] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.653853] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.654026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.654108] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.654260] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-08b77cbe-1697-46da-85d2-54fa8e8cc834 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.654360] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.654439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.654618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.654695] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.654803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.654997] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.654997] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.655267] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.655267] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('08b77cbe-1697-46da-85d2-54fa8e8cc834',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  833.655420] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 62cd528ac0e14f2e82538779976454da in queue reply_156db6c4256a4c58884afce15e6f5340
[  833.667819] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 62cd528ac0e14f2e82538779976454da
[  833.745283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  833.745777] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  833.745777] env[60890]: ERROR oslo_messaging.rpc.server 
[  833.898920] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.899133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.899261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.899521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.899664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.899986] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.900180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.900361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.900510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.900752] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.900917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.901052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.901260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.901394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.901544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.901789] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.901948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.902086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.902289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.902418] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.902650] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.902861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.903006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.903225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.903359] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  833.903587] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7efa8721925f439da60e7237c86f4555 in queue reply_156db6c4256a4c58884afce15e6f5340
[  833.911197] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7efa8721925f439da60e7237c86f4555
[  833.993778] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.993993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.994150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.994382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.994527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.994810] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.994975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.995098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.995310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.996081] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.996257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.996378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.996591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.996737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.996885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.997140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.997296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.997415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.997621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.997755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  833.998441] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  833.998612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  833.998737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  833.999007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  833.999181] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  833.999416] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 76eeddf1959e48f98d351baf222dd927 in queue reply_156db6c4256a4c58884afce15e6f5340
[  834.004757] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 76eeddf1959e48f98d351baf222dd927
[  834.158460] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.158460] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.158460] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 already deleted.
[  834.158460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.318385] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.318385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.318385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.318771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.318771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.319536] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.319536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.319536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.319536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.319777] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.319933] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.320057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.320261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.320398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.320542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.320798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.320953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.321086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.321275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.321402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.321633] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.322177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.322327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.322923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.322923] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  834.322923] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 944dcbf5e3b04b6aa4100334cc64cfa3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  834.327811] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 944dcbf5e3b04b6aa4100334cc64cfa3
[  834.407012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  834.407510] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  834.407510] env[60890]: ERROR oslo_messaging.rpc.server 
[  834.501453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  834.502129] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  834.502129] env[60890]: ERROR oslo_messaging.rpc.server 
[  834.508084] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ece24c3a88c14ca0ac3515e038f7e417 in queue reply_156db6c4256a4c58884afce15e6f5340
[  834.520558] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ece24c3a88c14ca0ac3515e038f7e417
[  834.825328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  834.825962] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  834.825962] env[60890]: ERROR oslo_messaging.rpc.server 
[  834.996521] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.996768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.996904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.997257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.997428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.997699] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.997864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.997988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.998221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.998469] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.998629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.998750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  834.998964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  834.999112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.999262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  834.999512] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  834.999671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  834.999794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.000023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.000170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.000411] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.000570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.000704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.000918] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.001086] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  835.001294] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 966ea46028fe418bbfc4b93e960321b9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  835.007344] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 966ea46028fe418bbfc4b93e960321b9
[  835.079322] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.079524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.079654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.079923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.080090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.080351] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.080509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.080629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.080844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.081349] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.081517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.081645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.081860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.082009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.082160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.082419] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.082577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.082698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.082935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.083080] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.083421] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.083587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.083710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.083924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.084076] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  835.084310] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ea0c4ae8600745ee8f6c6e13dd681e51 in queue reply_156db6c4256a4c58884afce15e6f5340
[  835.089330] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ea0c4ae8600745ee8f6c6e13dd681e51
[  835.433225] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.433549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.434118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.434118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.434312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.435305] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.435305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.435305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.435305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.435554] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.436022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.436022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.436148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.436678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.436678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.436678] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.436875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.436917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.437094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.437221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.437454] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  835.437601] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  835.437713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  835.437913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  835.438043] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  835.438275] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5e5cbf06a9c1462287d8648c56cb40ae in queue reply_156db6c4256a4c58884afce15e6f5340
[  835.443170] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5e5cbf06a9c1462287d8648c56cb40ae
[  835.502962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  835.503471] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  835.503471] env[60890]: ERROR oslo_messaging.rpc.server 
[  835.585988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  835.586520] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  835.586520] env[60890]: ERROR oslo_messaging.rpc.server 
[  835.940551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  835.941069] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  835.941069] env[60890]: ERROR oslo_messaging.rpc.server 
[  836.434671] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.435021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.435021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.435294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.435445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.435709] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.435907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.436062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.436299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.436551] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.436710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.436832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.437052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.437199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.437354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.437605] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.437761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.437883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.438097] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.438231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.438472] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.438629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.438747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.438957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.439096] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  836.439333] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 91109ac8f00243da84ed8868fbe7b5cd in queue reply_156db6c4256a4c58884afce15e6f5340
[  836.445540] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 91109ac8f00243da84ed8868fbe7b5cd
[  836.462199] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.462391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.462519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.462774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.462915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.463169] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.463329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.463450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.463667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.463913] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.464084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.464207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.464869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.464869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.464869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.465025] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.465150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.465290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.465473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.465633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.466264] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.466264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.466264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.466672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.466672] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  836.466672] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 71c7543471554c51a4987604439e714a in queue reply_156db6c4256a4c58884afce15e6f5340
[  836.471893] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 71c7543471554c51a4987604439e714a
[  836.495570] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.495960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.495960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.496940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.496940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.496940] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.496940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.497126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.500577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.500577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.500577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  836.500577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  836.500577] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  836.501297] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 17c3cd1e0f10413d821fcfc4c1df82b2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  836.505740] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 17c3cd1e0f10413d821fcfc4c1df82b2
[  836.942202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  836.942809] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  836.942809] env[60890]: ERROR oslo_messaging.rpc.server 
[  836.968820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  836.969325] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  836.969325] env[60890]: ERROR oslo_messaging.rpc.server 
[  837.002459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  837.002994] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  837.002994] env[60890]: ERROR oslo_messaging.rpc.server 
[  837.627951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.628328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.628381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.628633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.628780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.629051] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.629211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.629330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.629807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.629988] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.630047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.630128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.630344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.630487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.630685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.630890] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.631084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.631213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.631424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.631557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.631811] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.631965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.632104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.636815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.637024] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  837.637383] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e983959d2ed34e178743a06cb42df795 in queue reply_156db6c4256a4c58884afce15e6f5340
[  837.644988] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e983959d2ed34e178743a06cb42df795
[  837.667311] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.667520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.667648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.667919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.668084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.668353] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.668536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.668713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.668955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.669209] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.669881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.670027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.670287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.670503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.670587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.670844] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.671004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.671124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.671341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.671477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.671748] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.671922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.672057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.672272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.672407] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  837.672690] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5550b6a32d8a459aa98bf4428b4356e2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  837.678972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.678972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.678972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.678972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.678972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.678972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-485 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.679240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.679240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.679633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.679720] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.679884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.680029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.680253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.680427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.680705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-485" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.680881] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.681040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.681217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.681450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.681623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  837.681826] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-7e216a9e-b219-4c88-9332-01e34e59f8c2 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  837.681984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  837.682102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  837.682356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  837.682500] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e216a9e-b219-4c88-9332-01e34e59f8c2',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  837.682817] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4eb991721e7b431b81a72def0971ee57 in queue reply_156db6c4256a4c58884afce15e6f5340
[  837.683931] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5550b6a32d8a459aa98bf4428b4356e2
[  837.690644] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4eb991721e7b431b81a72def0971ee57
[  838.141923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  838.141923] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  838.141923] env[60890]: ERROR oslo_messaging.rpc.server 
[  838.176211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  838.176211] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  838.176211] env[60890]: ERROR oslo_messaging.rpc.server 
[  838.187744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-7e216a9e-b219-4c88-9332-01e34e59f8c2" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  838.187744] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  838.187744] env[60890]: ERROR oslo_messaging.rpc.server 
[  838.376283] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.377246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.377246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.377246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.377246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.377448] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.377448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.377621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.377781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.378025] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.378178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.378296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.378503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.378645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.378793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.379829] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.380071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.380200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.380586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.380586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.380782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.380984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.381120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.381327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.381469] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  838.381703] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a56253d42bc143ad921bd1b1cc4ba288 in queue reply_156db6c4256a4c58884afce15e6f5340
[  838.387144] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a56253d42bc143ad921bd1b1cc4ba288
[  838.654449] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  838.654812] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  838.868055] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.868284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.868417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.868708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.868863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.869128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.869288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.869408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.869628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.869879] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.870040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.870161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.870376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.870521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.870673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.870926] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.871086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.871206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.871418] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.871555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.871798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-311c73eb-96ce-4b32-b306-d58a365fa00f taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.871955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.877107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.877451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.877673] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('311c73eb-96ce-4b32-b306-d58a365fa00f',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  838.877984] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 09ee3fdd5ad64bc28e0b9e6be8ef7887 in queue reply_156db6c4256a4c58884afce15e6f5340
[  838.883566] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 09ee3fdd5ad64bc28e0b9e6be8ef7887
[  838.885228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  838.885743] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  838.885743] env[60890]: ERROR oslo_messaging.rpc.server 
[  838.934663] env[60890]: INFO networking_nsxv3.common.synchronization [req-c054220d-8cf4-45c3-9d89-8b4cac76bbc1 req-a0001df7-abf9-4a5e-9d6f-6bcdd87c6089 tempest-ServerActionsTestJSON-983883571 tempest-ServerActionsTestJSON-983883571-project-member] Enqueued Resource ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 with Priority: HIGHEST for action security_group_members
[  838.936810] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 with Priority: HIGHEST for action security_group_members
[  838.938533] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.938736] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.938904] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.940228] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-68938528-e0ea-4599-af8e-22dc66ee1e55 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.940228] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  838.940228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  838.946268] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('68938528-e0ea-4599-af8e-22dc66ee1e55',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  838.946268] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3cb3cfe716604b3aaa915d32395f9935 in queue reply_156db6c4256a4c58884afce15e6f5340
[  838.963372] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3cb3cfe716604b3aaa915d32395f9935
[  839.380669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-311c73eb-96ce-4b32-b306-d58a365fa00f" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  839.381197] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  839.381197] env[60890]: ERROR oslo_messaging.rpc.server 
[  839.443528] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.443748] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.443797] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.443987] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.444120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.444307] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.444412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.444460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.444612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.444691] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.444825] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 68938528-e0ea-4599-af8e-22dc66ee1e55 already deleted.
[  839.444992] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-68938528-e0ea-4599-af8e-22dc66ee1e55" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.705318] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.705629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.705629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.705915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.706056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.706318] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.706483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.706589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.706808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.707048] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.707202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.707320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.707528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.707691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.707829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.708104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.708259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.708382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.708583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.708716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.708952] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.709105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.709219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.709421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.709624] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  839.709809] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cd909017ec7b4ba9bdf47679d0b5b61f in queue reply_156db6c4256a4c58884afce15e6f5340
[  839.719819] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cd909017ec7b4ba9bdf47679d0b5b61f
[  839.975453] env[60890]: INFO networking_nsxv3.common.synchronization [req-029e1a92-fa8a-479e-8def-142d91df8a7a req-fa3aa081-a87f-4413-9053-cc6aec225d0b tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  839.975952] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  839.976311] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.976444] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.976488] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.976690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.976783] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.976949] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.977033] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.977093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.977280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.977437] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  839.977567] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  839.977611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  839.977762] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  839.977852] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  839.977958] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  839.978180] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 00c34d1f711d497cb596d8be619f29e7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  839.986126] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 00c34d1f711d497cb596d8be619f29e7
[  840.212362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  840.212911] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  840.212911] env[60890]: ERROR oslo_messaging.rpc.server 
[  840.400388] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.400599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.400741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.401010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.401165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.401430] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.401584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.401703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.401923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.402167] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.402323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.402442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.402645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.402788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.402938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.403421] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.403421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.403663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.403703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.403822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.404094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.404221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.404342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.404612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.404751] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  840.405039] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a27a40e209204d1db4fb027e8bd633bc in queue reply_156db6c4256a4c58884afce15e6f5340
[  840.410538] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a27a40e209204d1db4fb027e8bd633bc
[  840.480388] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.480529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.480600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.480991] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.481101] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.481294] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.481399] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.481460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.481616] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.481692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.481780] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  840.481857] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.648997] env[60890]: INFO networking_nsxv3.common.synchronization [None req-a9370a94-c4d8-46fd-89ba-f8ad0d489a8b None None] Enqueued Resource ID: bc56fe6d-b5a7-43e8-9887-06571e5caef1 with Priority: HIGHEST for action port
[  840.649255] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: bc56fe6d-b5a7-43e8-9887-06571e5caef1 with Priority: HIGHEST for action port
[  840.649830] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.649955] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.650022] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.650222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.650309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.650462] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-bc56fe6d-b5a7-43e8-9887-06571e5caef1 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.650560] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-bc56fe6d-b5a7-43e8-9887-06571e5caef1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.650621] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-bc56fe6d-b5a7-43e8-9887-06571e5caef1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.650828] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-bc56fe6d-b5a7-43e8-9887-06571e5caef1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.650909] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('bc56fe6d-b5a7-43e8-9887-06571e5caef1',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  840.651089] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 000524fdc3c746a9a07f90a36bd085b6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  840.656151] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 000524fdc3c746a9a07f90a36bd085b6
[  840.795221] env[60890]: INFO networking_nsxv3.common.synchronization [req-23fe4620-b919-4e16-9f81-4a57d6a3c953 req-2952e05e-ad53-4b95-b15b-39867cc89e8f tempest-MultipleCreateTestJSON-1912904963 tempest-MultipleCreateTestJSON-1912904963-project-member] Enqueued Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  840.795609] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  840.795955] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.796093] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.796159] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.796352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.796433] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.796577] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-08b77cbe-1697-46da-85d2-54fa8e8cc834 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.796667] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.796725] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.796867] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.796998] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.797085] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.797144] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.797275] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.797901] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.798035] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('08b77cbe-1697-46da-85d2-54fa8e8cc834',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  840.798227] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5c16d8774bc040dca7326c141c01a5b0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  840.810729] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5c16d8774bc040dca7326c141c01a5b0
[  840.886042] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.886243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.886367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.886637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.886777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.887031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.887489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.887489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.887489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.887734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.887905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.888034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.888240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.888378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.888620] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.888881] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.889034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.889151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.889448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.889668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.890037] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  840.890283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  840.890655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  840.890811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  840.890989] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  840.891234] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8b98ea1db0ba44e19813a3f1893c0aee in queue reply_156db6c4256a4c58884afce15e6f5340
[  840.896873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8b98ea1db0ba44e19813a3f1893c0aee
[  840.906568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  840.907034] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  840.907034] env[60890]: ERROR oslo_messaging.rpc.server 
[  841.152742] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-bc56fe6d-b5a7-43e8-9887-06571e5caef1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.152986] env[60890]: Traceback (most recent call last):
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  841.153058] env[60890]:     timer()
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  841.153058] env[60890]:     cb(*args, **kw)
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  841.153058] env[60890]:     result = function(*args, **kwargs)
[  841.153058] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  841.153058] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  841.153058] env[60890]:     return method(*args, **kwargs)
[  841.153058] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  841.153058] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  841.153058] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  841.153058] env[60890]:     result = self.transport._send(
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  841.153058] env[60890]:     return self._driver.send(target, ctxt, message,
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  841.153058] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  841.153058] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  841.153058] env[60890]:     raise result
[  841.153058] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  841.153058] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  841.299930] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.300032] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.300071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.300291] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.300414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.300620] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.300752] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.300793] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.300943] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.301029] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.301099] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 already deleted.
[  841.301176] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.393224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  841.393820] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  841.393820] env[60890]: ERROR oslo_messaging.rpc.server 
[  841.516519] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.516738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.516867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.517154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.517310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.517579] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.517736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.517859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.518085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.518341] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.518505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.518635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.518846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.518995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.519153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.519403] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.519555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.519673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.519882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.520029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.520273] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.520427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.520546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.520756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.520889] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  841.521671] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8c24612541414de1a33197061ea3b92c in queue reply_156db6c4256a4c58884afce15e6f5340
[  841.528124] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8c24612541414de1a33197061ea3b92c
[  841.528820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  841.529263] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  841.529263] env[60890]: ERROR oslo_messaging.rpc.server 
[  841.703554] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.703765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.703897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.704206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.704383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.704620] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.704804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.704927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.705159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.705410] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.705567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.705687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.705904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.706046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.706197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.706448] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.706604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.706724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.706935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.707075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  841.707339] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  841.707499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  841.707631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  841.707829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  841.714370] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  841.714702] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c0142ca8284f409fa34e594338e7521b in queue reply_156db6c4256a4c58884afce15e6f5340
[  841.721965] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c0142ca8284f409fa34e594338e7521b
[  842.110026] env[60890]: INFO networking_nsxv3.common.synchronization [None req-91a1c9a2-caaa-4cef-8ba9-2956ae462691 tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 with Priority: HIGHEST for action security_group_rules
[  842.110543] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 with Priority: HIGHEST for action security_group_rules
[  842.110850] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.110964] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.111036] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.111378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.111378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.111481] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-b9ba6520-46ed-4da6-847f-1a8360a3f912 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.111539] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.111630] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.111796] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.111870] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('b9ba6520-46ed-4da6-847f-1a8360a3f912',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  842.112658] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f57f55080d804efc90ef8a00b0ad0428 in queue reply_156db6c4256a4c58884afce15e6f5340
[  842.122508] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f57f55080d804efc90ef8a00b0ad0428
[  842.139263] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.139459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.139584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.139858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.140053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.140306] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.140452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.140563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.140785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.141014] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.141162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.141276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.141479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.141612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.141755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.142000] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.142150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.142268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.142471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.142604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.143503] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.143503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.143503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.143503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.143503] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  842.143884] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 58757832f7584f1ba4f1b975c851435f in queue reply_156db6c4256a4c58884afce15e6f5340
[  842.151481] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 58757832f7584f1ba4f1b975c851435f
[  842.217630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  842.218411] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  842.218411] env[60890]: ERROR oslo_messaging.rpc.server 
[  842.286363] env[60890]: INFO networking_nsxv3.common.synchronization [None req-7be8a925-cd7b-4b21-8795-8a583f8d19a1 tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 with Priority: HIGHEST for action security_group_rules
[  842.286845] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 with Priority: HIGHEST for action security_group_rules
[  842.287178] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.287305] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.287367] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.287570] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.287653] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.287804] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-b9ba6520-46ed-4da6-847f-1a8360a3f912 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.287940] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.635341] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.635522] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.635593] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.635762] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.635870] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.635965] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 already deleted.
[  842.636159] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.636242] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.636408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.636488] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('b9ba6520-46ed-4da6-847f-1a8360a3f912',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  842.636686] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3f789369371849a0a88c01b675848430 in queue reply_156db6c4256a4c58884afce15e6f5340
[  842.646399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  842.647170] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  842.647170] env[60890]: ERROR oslo_messaging.rpc.server 
[  842.660548] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3f789369371849a0a88c01b675848430
[  842.867140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.867140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.867140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.867140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.867140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.867941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.867941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.867941] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.867941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.867941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.868177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.868221] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  842.868510] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9a4e00cfbb5043b69375cc144a7b4e76 in queue reply_156db6c4256a4c58884afce15e6f5340
[  842.873985] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9a4e00cfbb5043b69375cc144a7b4e76
[  842.918791] env[60890]: INFO networking_nsxv3.common.synchronization [req-ca163bbc-c352-4f77-9900-ee19fd7c7966 req-c852a601-182b-4a62-97e1-26e2a7f09cb6 tempest-ServerGroupTestJSON-1644572680 tempest-ServerGroupTestJSON-1644572680-project-member] Enqueued Resource ID: 155cdf1b-11ca-448b-9d16-9ac84e3e1fa5 with Priority: HIGHEST for action security_group_members
[  842.919312] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 155cdf1b-11ca-448b-9d16-9ac84e3e1fa5 with Priority: HIGHEST for action security_group_members
[  842.919631] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.919758] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.919826] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.920080] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.920170] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.920330] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.921475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.921475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.921475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.921475] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  842.921475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  842.921475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  842.921475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  842.921475] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  842.921475] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('155cdf1b-11ca-448b-9d16-9ac84e3e1fa5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  842.921475] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ae1724d38d244ff5b4e1e557e773507f in queue reply_156db6c4256a4c58884afce15e6f5340
[  843.029576] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ae1724d38d244ff5b4e1e557e773507f
[  843.157786] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.158295] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.158295] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.158295] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.158496] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.158496] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 already deleted.
[  843.158615] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.203094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.203094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.203094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.203094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.203094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.203094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.205561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.205561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.205561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.205561] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  843.205561] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2a1d13603fde4430865a572306cf5e8b in queue reply_156db6c4256a4c58884afce15e6f5340
[  843.242598] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2a1d13603fde4430865a572306cf5e8b
[  843.251527] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.251743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.251873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.252166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.252619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.253068] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.253302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.253494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.253830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.254243] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.254474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.254652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.254993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.255204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.255429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.255741] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.256056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.256249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.256518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.256714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.257014] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.257249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.257423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.257716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.257918] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  843.258211] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5ee13b7a8df84dd58dcbd481d137ad1c in queue reply_156db6c4256a4c58884afce15e6f5340
[  843.263931] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5ee13b7a8df84dd58dcbd481d137ad1c
[  843.294967] env[60890]: INFO networking_nsxv3.common.synchronization [None req-a1e522f8-7b2b-449c-8e9c-76db696baf2d None None] Enqueued Resource ID: f2fae263-3b5e-4236-82ee-10c2540e1f11 with Priority: HIGHEST for action port
[  843.294967] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f2fae263-3b5e-4236-82ee-10c2540e1f11 with Priority: HIGHEST for action port
[  843.295584] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.295776] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.295917] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.296199] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.296334] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.296527] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-f2fae263-3b5e-4236-82ee-10c2540e1f11 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.296666] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-f2fae263-3b5e-4236-82ee-10c2540e1f11" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.296760] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-f2fae263-3b5e-4236-82ee-10c2540e1f11" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.296997] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-f2fae263-3b5e-4236-82ee-10c2540e1f11" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.297160] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f2fae263-3b5e-4236-82ee-10c2540e1f11',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  843.298969] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9c70b427b66b4e70876efe50c0fb929e in queue reply_156db6c4256a4c58884afce15e6f5340
[  843.325570] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9c70b427b66b4e70876efe50c0fb929e
[  843.370563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  843.371048] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  843.371048] env[60890]: ERROR oslo_messaging.rpc.server 
[  843.423905] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.424087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.424163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.424374] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.424467] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.424655] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.424754] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.424975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.425166] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.425257] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.425353] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 155cdf1b-11ca-448b-9d16-9ac84e3e1fa5 already deleted.
[  843.425438] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.707253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  843.707801] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  843.707801] env[60890]: ERROR oslo_messaging.rpc.server 
[  843.760689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  843.761201] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  843.761201] env[60890]: ERROR oslo_messaging.rpc.server 
[  843.801436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-f2fae263-3b5e-4236-82ee-10c2540e1f11" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.802512] env[60890]: Traceback (most recent call last):
[  843.802554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  843.802554] env[60890]:     timer()
[  843.802554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  843.802554] env[60890]:     cb(*args, **kw)
[  843.802554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  843.802554] env[60890]:     result = function(*args, **kwargs)
[  843.802554] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  843.802554] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  843.802554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  843.802554] env[60890]:     return method(*args, **kwargs)
[  843.802554] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  843.802554] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  843.802914] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  843.802914] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  843.802914] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  843.802914] env[60890]:     result = self.transport._send(
[  843.802914] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  843.802914] env[60890]:     return self._driver.send(target, ctxt, message,
[  843.802914] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  843.802914] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  843.802914] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  843.802914] env[60890]:     raise result
[  843.802914] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  843.802914] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  843.956211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.956211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-568 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.956211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-568" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.956211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.956211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.959585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.959585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.959585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  843.959585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-2b0c270c-111e-4601-b891-f25a8b96f34e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  843.959585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  843.959585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  843.959585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  843.959585] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2b0c270c-111e-4601-b891-f25a8b96f34e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  843.959585] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e473837e7e594b61a4484274ddf08303 in queue reply_156db6c4256a4c58884afce15e6f5340
[  843.963986] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e473837e7e594b61a4484274ddf08303
[  844.099814] env[60890]: INFO networking_nsxv3.common.synchronization [req-9f627974-60b5-4250-8fba-079d1fa0bcaf req-43caa1f9-6f6a-4aef-a70f-4fed0b3b5293 tempest-ServerRescueTestJSONUnderV235-1496514119 tempest-ServerRescueTestJSONUnderV235-1496514119-project-member] Enqueued Resource ID: abb765ab-3f60-4c3f-9229-badb809c9b99 with Priority: HIGHEST for action security_group_members
[  844.101163] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: abb765ab-3f60-4c3f-9229-badb809c9b99 with Priority: HIGHEST for action security_group_members
[  844.101641] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.101947] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.102146] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.102460] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.102812] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.103192] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-abb765ab-3f60-4c3f-9229-badb809c9b99 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.103451] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-abb765ab-3f60-4c3f-9229-badb809c9b99" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.103638] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-abb765ab-3f60-4c3f-9229-badb809c9b99" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.103979] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-abb765ab-3f60-4c3f-9229-badb809c9b99" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.104300] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.104547] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.104732] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.105022] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.105333] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.105557] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('abb765ab-3f60-4c3f-9229-badb809c9b99',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  844.105879] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9895a9432f77456aad8a53e6e8134640 in queue reply_156db6c4256a4c58884afce15e6f5340
[  844.115892] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9895a9432f77456aad8a53e6e8134640
[  844.255485] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.255815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.255815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.256121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.256269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.256523] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.256675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.256790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.257015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.257259] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.257410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.257524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.257730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.257877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.258024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.258276] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.258428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.258541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.258749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.258876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.259112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.259258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.259374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.259580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.259708] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  844.259934] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 89fff51edc7e4273a87edede2169e93d in queue reply_156db6c4256a4c58884afce15e6f5340
[  844.266098] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 89fff51edc7e4273a87edede2169e93d
[  844.346096] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.346291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.346417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.346689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.346841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.347096] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.347265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.347442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.347662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.347904] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.348074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.348337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.348412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.348527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.348681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.348969] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.349132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.349245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.349452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.349584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.349831] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.349982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.350136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.350360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.350492] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  844.350721] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8e910b120d954f509d494a3acf1a9d48 in queue reply_156db6c4256a4c58884afce15e6f5340
[  844.356161] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8e910b120d954f509d494a3acf1a9d48
[  844.427546] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.427746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.427874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.428163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.428312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.428570] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.428727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.428846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.429160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.429372] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.429529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.429648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.429903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.430075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.430229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.430477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.430632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.430749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.430985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.431233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.431508] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.431668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.431815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.432085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.432229] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  844.432465] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8c5158461dd94f9fb8646451f1663cf6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  844.438720] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8c5158461dd94f9fb8646451f1663cf6
[  844.460273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-2b0c270c-111e-4601-b891-f25a8b96f34e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  844.460803] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  844.460803] env[60890]: ERROR oslo_messaging.rpc.server 
[  844.609332] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.609528] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.609616] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.609835] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.609928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.610135] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  844.610234] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  844.610297] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  844.610440] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  844.610527] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.610616] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: abb765ab-3f60-4c3f-9229-badb809c9b99 already deleted.
[  844.610699] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-abb765ab-3f60-4c3f-9229-badb809c9b99" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.762534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  844.763053] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  844.763053] env[60890]: ERROR oslo_messaging.rpc.server 
[  844.853251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  844.853761] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  844.853761] env[60890]: ERROR oslo_messaging.rpc.server 
[  844.934651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  844.935229] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  844.935229] env[60890]: ERROR oslo_messaging.rpc.server 
[  845.203637] env[60890]: INFO networking_nsxv3.common.synchronization [req-23fe4620-b919-4e16-9f81-4a57d6a3c953 req-005efb98-83a9-471c-a577-16485ed236d3 tempest-MultipleCreateTestJSON-1912904963 tempest-MultipleCreateTestJSON-1912904963-project-member] Enqueued Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  845.204189] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 with Priority: HIGHEST for action security_group_members
[  845.204556] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.204662] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.204712] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.204951] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.205066] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.205210] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-08b77cbe-1697-46da-85d2-54fa8e8cc834 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.205299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.205369] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.205516] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.205664] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.205751] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.205821] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.205987] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.206069] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.206159] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('08b77cbe-1697-46da-85d2-54fa8e8cc834',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  845.206374] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 685a328a59454ee3819f628b6bc5bdcf in queue reply_156db6c4256a4c58884afce15e6f5340
[  845.218185] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 685a328a59454ee3819f628b6bc5bdcf
[  845.313351] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.313351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.313351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.314484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.314693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.315034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.315223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.315348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.315581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.315869] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.316062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.316187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.316405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.316551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.316703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.316964] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.317134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.317279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.317465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.317597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.317850] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.318003] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.318123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.318333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.318469] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  845.318696] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 40ff662513fa4123965c64c3b41fa8be in queue reply_156db6c4256a4c58884afce15e6f5340
[  845.323565] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 40ff662513fa4123965c64c3b41fa8be
[  845.389989] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.390193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.390324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.390591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.390923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.391217] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.391380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.391501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.391730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.392100] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.392277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.392398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.392611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.392798] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.392978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.393235] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.393394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.393516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.393725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.393861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.394100] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.394255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.394374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.394581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.394716] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  845.395030] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 225c4ec432b94e9e8def10b0d425182a in queue reply_156db6c4256a4c58884afce15e6f5340
[  845.400205] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 225c4ec432b94e9e8def10b0d425182a
[  845.473257] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.473521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.473654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.474010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.474094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.474634] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.474855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.474992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.475222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.475478] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.475638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.475759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.475991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.476152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.476304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.476555] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.476713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.476833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.477043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.477179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.477417] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.477574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.477692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.477901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.478037] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  845.478273] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d1aaf0821c5e48ad8f7fac278fc00e46 in queue reply_156db6c4256a4c58884afce15e6f5340
[  845.482667] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d1aaf0821c5e48ad8f7fac278fc00e46
[  845.709387] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.709387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.709387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.709387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.709387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.709387] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  845.709387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  845.709387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  845.709781] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  845.709781] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.709781] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 08b77cbe-1697-46da-85d2-54fa8e8cc834 already deleted.
[  845.709781] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-08b77cbe-1697-46da-85d2-54fa8e8cc834" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.820634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  845.821161] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  845.821161] env[60890]: ERROR oslo_messaging.rpc.server 
[  845.897636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  845.898173] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  845.898173] env[60890]: ERROR oslo_messaging.rpc.server 
[  845.984462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  845.984462] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  845.984462] env[60890]: ERROR oslo_messaging.rpc.server 
[  846.426449] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.426776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.426852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.427064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.427225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.427526] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.427648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.427749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.427971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.428320] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.428431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.428544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.428759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.428914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.429154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.429315] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.429471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.429591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.429804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.429944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.430184] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.430339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.430459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.430672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.430851] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  846.431047] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 00b9acec75b24f06abf3881e155c11b3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  846.433601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.434012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.434012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.434135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.434218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.434464] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.434621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.436625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.436625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.436625] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.436625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.436625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.436625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.436625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.436625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.437058] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.437058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.437058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.437273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.437411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.437655] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.437814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.437938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.438151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.438288] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  846.438505] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 411926ab9bc841f385ceeebf6e689025 in queue reply_156db6c4256a4c58884afce15e6f5340
[  846.439366] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 00b9acec75b24f06abf3881e155c11b3
[  846.442592] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 411926ab9bc841f385ceeebf6e689025
[  846.514760] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.515035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.515165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.515414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.515558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.515897] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.516076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.516198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.516411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.516651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.516857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.516931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.517126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.517280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.517428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.517673] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.517824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.517943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.518146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.518276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.518509] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  846.518661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  846.518777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  846.518981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  846.519112] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  846.519328] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a75b282cbe404dc5be9f69792dbdf928 in queue reply_156db6c4256a4c58884afce15e6f5340
[  846.524407] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a75b282cbe404dc5be9f69792dbdf928
[  846.933099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  846.933618] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  846.933618] env[60890]: ERROR oslo_messaging.rpc.server 
[  846.940419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  846.940884] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  846.940884] env[60890]: ERROR oslo_messaging.rpc.server 
[  847.021425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  847.021908] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  847.021908] env[60890]: ERROR oslo_messaging.rpc.server 
[  847.478044] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.478337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.478391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.478646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.478889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.479141] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.479302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.479421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.479636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.479882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.480053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.480176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.480385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.480525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.480672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.480950] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.481121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.481246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.481452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.481586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.481841] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.481999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.482117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.482323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.482459] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  847.482687] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6e5584d0355d404fbf17d985ceff91cb in queue reply_156db6c4256a4c58884afce15e6f5340
[  847.487643] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6e5584d0355d404fbf17d985ceff91cb
[  847.559473] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.559678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.559797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.560378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.560378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.560535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.560631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.560756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.560968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.561213] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.561367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.561485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.561852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.561852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.562016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.562200] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.562354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.562464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.562790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.562856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.563061] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.563213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.563371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.563527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.563663] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  847.563957] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a7f326a8bdce46b1826518d41bece13f in queue reply_156db6c4256a4c58884afce15e6f5340
[  847.569639] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a7f326a8bdce46b1826518d41bece13f
[  847.663893] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.665227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.665227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.665227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.665227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.665227] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.665515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.665515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.665643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.665966] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.666108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.666318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.666532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.666674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.666831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.667608] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.667779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.667909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.668137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.668306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.668560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  847.668718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  847.668843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  847.669043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  847.669180] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  847.669405] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 68f9818106524f64a902a68d7587082f in queue reply_156db6c4256a4c58884afce15e6f5340
[  847.674124] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 68f9818106524f64a902a68d7587082f
[  847.984753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  847.985331] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  847.985331] env[60890]: ERROR oslo_messaging.rpc.server 
[  848.066727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  848.067230] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  848.067230] env[60890]: ERROR oslo_messaging.rpc.server 
[  848.165908] env[60890]: INFO networking_nsxv3.common.synchronization [req-c741c015-96e7-4d6b-9b66-799c654a1eb3 req-b5078592-a039-4ba5-bc9a-3685876e7844 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  848.166415] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  848.166734] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.166858] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.166929] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.167138] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.167230] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.167386] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.167482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.167555] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.167709] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.167849] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.167940] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.167999] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.168201] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.168244] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.168305] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  848.168480] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 36651ed5b852452688829620b0d4db29 in queue reply_156db6c4256a4c58884afce15e6f5340
[  848.170845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  848.171270] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  848.171270] env[60890]: ERROR oslo_messaging.rpc.server 
[  848.178490] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 36651ed5b852452688829620b0d4db29
[  848.456804] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.457144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.457270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.457646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.457847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.458189] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.458403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.458563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.458851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.459193] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.459386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.459512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.459724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.459867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.460030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.460287] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.460444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.460563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.460772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.460909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.461145] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.461298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.461418] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.461687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.461830] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  848.462066] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d16f0edbd72c44d89f57d3368c226e93 in queue reply_156db6c4256a4c58884afce15e6f5340
[  848.467643] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d16f0edbd72c44d89f57d3368c226e93
[  848.522871] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.523145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.523191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.523443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.523585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.523835] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.523994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.524129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.524342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.524585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.524739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.524883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.525140] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.525231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.525380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.525630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.525794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.525908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.526113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.526249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.526483] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.526638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.526755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.526958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.527120] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  848.527367] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 79cd6f00f86f4ff7b3a8518499197d6b in queue reply_156db6c4256a4c58884afce15e6f5340
[  848.532662] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 79cd6f00f86f4ff7b3a8518499197d6b
[  848.606126] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.606322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.606447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.606725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.606880] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.607147] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.607298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.607412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.607625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.607873] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.608055] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.608175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.608377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.608514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.608656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.608902] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.609047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.609161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.609362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.609489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.609721] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.609875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.609988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.610189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.610319] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  848.610546] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg df5da83580e54f46832b83f5a6070056 in queue reply_156db6c4256a4c58884afce15e6f5340
[  848.616350] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg df5da83580e54f46832b83f5a6070056
[  848.670780] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.670971] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.671020] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.671213] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.671303] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.671967] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.671967] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.671967] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.671967] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.671967] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.671967] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  848.672518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.742746] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.742746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.742746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.742746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.742746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.742746] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.742746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.742746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.743335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.743335] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.743581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.743581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.743888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.743888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.744016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.744276] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.744363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.744489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.744697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.744855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.745100] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  848.745257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  848.745375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  848.745600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  848.745715] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  848.745953] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7d631afbea9e4b45b1a35396eeffe148 in queue reply_156db6c4256a4c58884afce15e6f5340
[  848.753369] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7d631afbea9e4b45b1a35396eeffe148
[  848.964423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  848.964964] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  848.964964] env[60890]: ERROR oslo_messaging.rpc.server 
[  849.029594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  849.030066] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  849.030066] env[60890]: ERROR oslo_messaging.rpc.server 
[  849.112409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  849.112948] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  849.112948] env[60890]: ERROR oslo_messaging.rpc.server 
[  849.247883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  849.248395] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  849.248395] env[60890]: ERROR oslo_messaging.rpc.server 
[  849.567912] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.568212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.568264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.568532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.568679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.568973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.569138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.569259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.569482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.569726] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.569885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.570005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.570215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.570355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.570507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.570757] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.570914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.571035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.571242] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.571377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.571617] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.571786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.571920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.573176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.573176] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  849.573176] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f6c047a98b0248299596f315fa8d9f03 in queue reply_156db6c4256a4c58884afce15e6f5340
[  849.577596] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f6c047a98b0248299596f315fa8d9f03
[  849.651581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.651818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.651950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.652291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.652447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.652716] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.653034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.653181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.653422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.653688] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.653855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.654039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.654197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.654367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.654524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.654823] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.654996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.655120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.655336] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.655472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.655709] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.655885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.656054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.656282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.656421] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  849.657194] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 81144e02a8f24b9c8dda98b533739b77 in queue reply_156db6c4256a4c58884afce15e6f5340
[  849.661815] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 81144e02a8f24b9c8dda98b533739b77
[  849.662618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.662795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.662919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.663137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.663314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.663608] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.663792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.663917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.664144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.664397] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.664555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.664705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.664945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.665091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.665241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.665490] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.665647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.665768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.665998] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.666159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.666406] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.666564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.666682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.666891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.667026] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  849.667242] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 38e8ff41e58948469bf639daeb7571b7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  849.671415] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 38e8ff41e58948469bf639daeb7571b7
[  849.881671] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.881882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.882009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.882277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.882419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.882679] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.882860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.882981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.883200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.883444] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.883600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.883714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.883920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.884075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.884228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.884480] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.884635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.884755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.885041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.885195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  849.885450] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4bcaf119-a845-4821-8b67-0fa4ac66ad43 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  849.885606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  849.885725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  849.886021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  849.886160] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4bcaf119-a845-4821-8b67-0fa4ac66ad43',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  849.886395] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 388fab809ad44b3885960b41747b88da in queue reply_156db6c4256a4c58884afce15e6f5340
[  849.891333] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 388fab809ad44b3885960b41747b88da
[  850.074402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.074956] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.074956] env[60890]: ERROR oslo_messaging.rpc.server 
[  850.158819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.163715] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.163715] env[60890]: ERROR oslo_messaging.rpc.server 
[  850.168990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.169497] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.169497] env[60890]: ERROR oslo_messaging.rpc.server 
[  850.388586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4bcaf119-a845-4821-8b67-0fa4ac66ad43" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.389050] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.389050] env[60890]: ERROR oslo_messaging.rpc.server 
[  850.650757] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.651062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.651124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.651343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.651485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.651747] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.652078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.652078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.652253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.652558] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.652745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.653539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.653539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.653539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.653539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.654016] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.654016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.654016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.654310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.654310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.654528] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.654681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.654853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.655075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.655213] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  850.655444] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e5eb57e8525848f6b104699ab1f4b313 in queue reply_156db6c4256a4c58884afce15e6f5340
[  850.660285] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e5eb57e8525848f6b104699ab1f4b313
[  850.728508] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.728703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.728828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.729092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.729231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.729480] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.729630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.729746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.729956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.730194] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.730344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.730460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.730661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.730804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.730948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.731188] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.731335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.731451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.731652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.731781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.732128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.732318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.732396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.732602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.733578] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  850.733834] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 233b202d309940509f247471869c8d94 in queue reply_156db6c4256a4c58884afce15e6f5340
[  850.735675] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.735867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.735996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.736245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.736381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.736619] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.736781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.736883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.737115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.737312] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.737460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.737573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.737772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.737903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.739016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.739016] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.739016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.739016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.739016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.739016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.739251] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  850.739251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  850.739345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  850.739536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  850.739668] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  850.739888] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d15a7d94a47d4c26b446c8d3b9e8c604 in queue reply_156db6c4256a4c58884afce15e6f5340
[  850.740763] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 233b202d309940509f247471869c8d94
[  850.741317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.741735] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  850.741735] env[60890]: ERROR oslo_messaging.rpc.server 
[  850.745290] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d15a7d94a47d4c26b446c8d3b9e8c604
[  851.158211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  851.158533] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  851.158533] env[60890]: ERROR oslo_messaging.rpc.server 
[  851.244292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  851.244854] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  851.244854] env[60890]: ERROR oslo_messaging.rpc.server 
[  851.458672] env[60890]: INFO networking_nsxv3.common.synchronization [req-5cb1652d-8f2e-4b5a-80ea-297775307349 req-660d6a12-b69b-4c43-a79b-4392c85bceae tempest-ServersNegativeTestJSON-229127054 tempest-ServersNegativeTestJSON-229127054-project-member] Enqueued Resource ID: 8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d with Priority: HIGHEST for action security_group_members
[  851.459190] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d with Priority: HIGHEST for action security_group_members
[  851.459517] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.459646] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.459706] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.459912] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.459996] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.460172] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.460263] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.460323] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.460526] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.460663] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.460752] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.460808] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.460942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.461017] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.461101] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  851.461278] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 24e7fb8586db40e7b3b194b8bdd8514a in queue reply_156db6c4256a4c58884afce15e6f5340
[  851.470044] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 24e7fb8586db40e7b3b194b8bdd8514a
[  851.960067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.960067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.960067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.960067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.960067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.960067] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  851.960067] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 544b92005e5248c4bc4c5a6e5a4e1174 in queue reply_156db6c4256a4c58884afce15e6f5340
[  851.960067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.960067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.960067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.960067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.964858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.965231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.965669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.981569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.982110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.982595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.982863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.983271] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.983585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.983817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.984170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.984452] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  851.984863] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 24aebb1c187b48179fd423afabb22f2d in queue reply_156db6c4256a4c58884afce15e6f5340
[  851.986430] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 544b92005e5248c4bc4c5a6e5a4e1174
[  851.988756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  851.990597] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  851.990597] env[60890]: ERROR oslo_messaging.rpc.server 
[  851.993297] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.993555] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.993728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.993994] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.994261] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.994580] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  851.994810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  851.994980] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  851.995499] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  851.995702] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.995915] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d already deleted.
[  851.996121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  851.997063] env[60890]: INFO networking_nsxv3.common.synchronization [req-029e1a92-fa8a-479e-8def-142d91df8a7a req-1205c24f-8c1e-4a7c-8ab6-639d2996b2f3 tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  851.997859] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 24aebb1c187b48179fd423afabb22f2d
[  852.000160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  852.000704] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  852.000704] env[60890]: ERROR oslo_messaging.rpc.server 
[  852.002949] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  852.004454] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.004698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.004883] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.010652] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.010652] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.010652] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.010652] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  852.010652] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 7c0bd69535894482a5bbea70b8474124 in queue reply_156db6c4256a4c58884afce15e6f5340
[  852.020073] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7c0bd69535894482a5bbea70b8474124
[  852.122510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.122510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.122510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.122510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.122510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.122510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.122510] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  852.122510] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg efb125b1760b414282d5e49654ca8097 in queue reply_156db6c4256a4c58884afce15e6f5340
[  852.129694] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg efb125b1760b414282d5e49654ca8097
[  852.248965] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.249171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.249300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.249573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.249719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.249979] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.250142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.250263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.250542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.250792] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.250947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.251068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.251275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.251421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.251571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.251820] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.251973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.252127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.252338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.252473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.252742] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.252965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.253086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.253338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.253481] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  852.253712] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7cb12628a529422981f8531745019743 in queue reply_156db6c4256a4c58884afce15e6f5340
[  852.259217] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7cb12628a529422981f8531745019743
[  852.513443] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.513612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.513683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.513888] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.513964] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.514158] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.514271] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.514329] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.514463] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.514527] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.514618] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  852.514689] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.524031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.524234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.524358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.524606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.524739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.525025] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.525175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.525428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.525667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.525931] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.526085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.526206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.526425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.526659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.526724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.526950] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.527105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.527221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.527504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.527654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.527981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.528195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.529015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.529015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.529015] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  852.529015] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6d13b7385f6e44928743ceb10daef476 in queue reply_156db6c4256a4c58884afce15e6f5340
[  852.531784] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.531980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.532094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.532322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.532462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.532710] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-352 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.532906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.534028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.534028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.534028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.534028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.534028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.534324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.534386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.534533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-352" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.534811] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.534959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.535083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.535284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.535528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.535809] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5bd51cef-28a3-43d0-adc6-c9a8516e583d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  852.535984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  852.536122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  852.536338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  852.536477] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5bd51cef-28a3-43d0-adc6-c9a8516e583d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  852.536946] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 18a990bdc2434482b02be776ae750ad6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  852.537631] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6d13b7385f6e44928743ceb10daef476
[  852.541355] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 18a990bdc2434482b02be776ae750ad6
[  852.624603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  852.625174] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  852.625174] env[60890]: ERROR oslo_messaging.rpc.server 
[  852.756717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  852.757475] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  852.757475] env[60890]: ERROR oslo_messaging.rpc.server 
[  853.031190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  853.031828] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  853.031828] env[60890]: ERROR oslo_messaging.rpc.server 
[  853.039645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5bd51cef-28a3-43d0-adc6-c9a8516e583d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  853.040416] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  853.040416] env[60890]: ERROR oslo_messaging.rpc.server 
[  853.284584] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.284855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.285016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.285286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.285440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.285704] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.286100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.286223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.286442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.286695] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.286906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.286963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.287177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.287322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.287471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.287724] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.287883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.288009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.288224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.288362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.288601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.288774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.289021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.289255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.289394] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  853.289627] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3822da18045a41abace1d38704624817 in queue reply_156db6c4256a4c58884afce15e6f5340
[  853.295105] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3822da18045a41abace1d38704624817
[  853.299312] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.299498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.299625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.299860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.300010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.300272] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.300430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.300551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.300759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.301034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.301197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.301317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.301524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.301664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.301809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.302058] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.302213] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.302331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.302536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.302705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.302906] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.303060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.303176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.303381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.303512] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  853.303735] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d50caa54166d4324ad2eb5be1cf2c0be in queue reply_156db6c4256a4c58884afce15e6f5340
[  853.309254] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d50caa54166d4324ad2eb5be1cf2c0be
[  853.675850] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.676104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.676242] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.676513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.676664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.676931] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-233 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.677092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.677210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.677427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.677672] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.677829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.677952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.678160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.678304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.678455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-233" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.678706] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.678959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.679103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.679318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.679459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.679699] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-f05b8916-788c-4b79-b795-6204993dd97a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.679854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.679973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.680197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.680335] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f05b8916-788c-4b79-b795-6204993dd97a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  853.680571] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5e1d3fd3a47b40b398f734428a536346 in queue reply_156db6c4256a4c58884afce15e6f5340
[  853.686503] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5e1d3fd3a47b40b398f734428a536346
[  853.791759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  853.792465] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  853.792465] env[60890]: ERROR oslo_messaging.rpc.server 
[  853.805288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  853.805737] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  853.805737] env[60890]: ERROR oslo_messaging.rpc.server 
[  853.864056] env[60890]: INFO networking_nsxv3.common.synchronization [req-ca163bbc-c352-4f77-9900-ee19fd7c7966 req-2d5ce7ae-93a8-495c-96fa-88ab555d2a7e tempest-ServerGroupTestJSON-1644572680 tempest-ServerGroupTestJSON-1644572680-project-member] Enqueued Resource ID: 155cdf1b-11ca-448b-9d16-9ac84e3e1fa5 with Priority: HIGHEST for action security_group_members
[  853.864571] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 155cdf1b-11ca-448b-9d16-9ac84e3e1fa5 with Priority: HIGHEST for action security_group_members
[  853.864930] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.865056] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.865119] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.865316] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.865403] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.865558] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.865678] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.865740] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.866076] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.866076] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  853.866191] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  853.866191] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  853.866289] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  853.866352] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  853.866446] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('155cdf1b-11ca-448b-9d16-9ac84e3e1fa5',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  853.866620] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 50499e9171ba4b73918ced1383d4f748 in queue reply_156db6c4256a4c58884afce15e6f5340
[  853.880090] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 50499e9171ba4b73918ced1383d4f748
[  854.183018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-f05b8916-788c-4b79-b795-6204993dd97a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  854.184836] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  854.184836] env[60890]: ERROR oslo_messaging.rpc.server 
[  854.361244] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.361431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.361555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.361833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.361978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.362230] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.362382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.362497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.362707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.362944] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.363093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.363206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.363408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.363544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.363685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.363930] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.364096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.364212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.364416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.364545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.364791] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.364963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.365077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.365279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.365409] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  854.365636] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 55ddcf807f144ce1bea331aac5acca87 in queue reply_156db6c4256a4c58884afce15e6f5340
[  854.370486] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.370597] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.370665] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.370815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.370924] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.371092] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.371185] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.371246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.371385] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.371459] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.371539] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 155cdf1b-11ca-448b-9d16-9ac84e3e1fa5 already deleted.
[  854.371613] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-155cdf1b-11ca-448b-9d16-9ac84e3e1fa5" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.372114] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 55ddcf807f144ce1bea331aac5acca87
[  854.446852] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.447049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.447173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.447518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.447663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.447943] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.448089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.448207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.448426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.452195] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.452399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.452538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.452769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.452928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.453083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.453345] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.453500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.453618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.453837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.453967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.454211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.454358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.454476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.454681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.454831] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  854.455069] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ad67e162efa04ec5886d669bc7e8ce6c in queue reply_156db6c4256a4c58884afce15e6f5340
[  854.471289] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ad67e162efa04ec5886d669bc7e8ce6c
[  854.868362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  854.868881] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  854.868881] env[60890]: ERROR oslo_messaging.rpc.server 
[  854.938050] env[60890]: INFO networking_nsxv3.common.synchronization [req-9f627974-60b5-4250-8fba-079d1fa0bcaf req-215142af-92e8-4cee-9073-136090b3c03e tempest-ServerRescueTestJSONUnderV235-1496514119 tempest-ServerRescueTestJSONUnderV235-1496514119-project-member] Enqueued Resource ID: abb765ab-3f60-4c3f-9229-badb809c9b99 with Priority: HIGHEST for action security_group_members
[  854.939014] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: abb765ab-3f60-4c3f-9229-badb809c9b99 with Priority: HIGHEST for action security_group_members
[  854.939240] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.939372] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.939442] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.939648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.939736] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.939890] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-abb765ab-3f60-4c3f-9229-badb809c9b99 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.939990] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-abb765ab-3f60-4c3f-9229-badb809c9b99" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.940071] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-abb765ab-3f60-4c3f-9229-badb809c9b99" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.940226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-abb765ab-3f60-4c3f-9229-badb809c9b99" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.940366] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  854.940459] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  854.940522] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  854.940660] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  854.940739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.940827] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('abb765ab-3f60-4c3f-9229-badb809c9b99',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  854.941212] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5ddb126f0a294a6294bffdbe0d9495f3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  854.948804] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5ddb126f0a294a6294bffdbe0d9495f3
[  854.965552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  854.965973] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  854.965973] env[60890]: ERROR oslo_messaging.rpc.server 
[  855.443728] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.444026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.444026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.444202] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.444300] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.444498] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.444601] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.444665] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.444903] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.444991] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.445080] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: abb765ab-3f60-4c3f-9229-badb809c9b99 already deleted.
[  855.445161] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-abb765ab-3f60-4c3f-9229-badb809c9b99" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.450299] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.450299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.450299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.450299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.450706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.450706] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.450787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.450887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.451095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.451339] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.451495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.451615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.451823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.451967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.452138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.452390] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.452545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.452750] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.452896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.453155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.453343] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.453469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.453560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.454089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.454089] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  855.454179] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9e1d7cf36e0c4864a52c70ba2a247402 in queue reply_156db6c4256a4c58884afce15e6f5340
[  855.460156] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9e1d7cf36e0c4864a52c70ba2a247402
[  855.592165] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.592348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.592478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.592751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.592900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.593158] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.593316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.593437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.593650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.593895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.594052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.594171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.594379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.594523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.594673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.595015] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.595176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.595296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.595507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.595645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.595915] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.596106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.596231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.596524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.596668] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  855.596904] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 759a0fdd4e6a40e39e0b547ddfd01573 in queue reply_156db6c4256a4c58884afce15e6f5340
[  855.602546] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 759a0fdd4e6a40e39e0b547ddfd01573
[  855.654298] env[60890]: INFO networking_nsxv3.common.synchronization [req-c0cafa2e-fafd-4848-9979-ccd2e8070f13 req-e8fb27fa-d23e-4fa4-b275-bd6f4227c901 tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb with Priority: HIGHEST for action security_group_members
[  855.654954] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb with Priority: HIGHEST for action security_group_members
[  855.655183] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.655314] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.655369] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.655569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.655665] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.655848] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-beabdba8-4b41-4b8f-a694-d75a0cc077bb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.655965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.656041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.656261] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.656407] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.656501] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.656563] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.656702] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.656782] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.656907] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('beabdba8-4b41-4b8f-a694-d75a0cc077bb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  855.657108] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg b174f030f7104fe595149f93fefd515e in queue reply_156db6c4256a4c58884afce15e6f5340
[  855.677867] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b174f030f7104fe595149f93fefd515e
[  855.913489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.913703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.913836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.914201] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.914280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.914541] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.914701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.914895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.915182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.915438] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.915600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.915721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.915934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.916091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.916249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.916501] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.916658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.916777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.917245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.917245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.917465] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  855.917622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  855.917741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  855.917988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  855.918127] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  855.918361] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 60bd1ab36db24c4d8dc1b0637abdd599 in queue reply_156db6c4256a4c58884afce15e6f5340
[  855.923050] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 60bd1ab36db24c4d8dc1b0637abdd599
[  855.955809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  855.956278] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  855.956278] env[60890]: ERROR oslo_messaging.rpc.server 
[  856.098804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  856.099273] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  856.099273] env[60890]: ERROR oslo_messaging.rpc.server 
[  856.161669] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.161820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.161894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.162195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.162301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.162504] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.162605] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.162669] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.162837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.162916] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.163001] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb already deleted.
[  856.163082] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.420233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  856.420747] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  856.420747] env[60890]: ERROR oslo_messaging.rpc.server 
[  856.495786] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.495786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.495786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.496174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.496264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.496546] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.496702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.496824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.497044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.497330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.497492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.497614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.497825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.498051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.498212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.498465] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.498622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.498741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.498969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.499106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.499344] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.499498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.499616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.499824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.499962] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  856.500205] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d2e7487a7c534f3dbaef25821272698b in queue reply_156db6c4256a4c58884afce15e6f5340
[  856.505306] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d2e7487a7c534f3dbaef25821272698b
[  856.652733] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.652987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.653124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.653395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.653546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.654715] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.654715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.654715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.654715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.654715] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.654715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.655010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.655096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.655243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.655398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.655652] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.655910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.656156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.656447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.656605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.656865] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.657027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.657170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.657390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.657533] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  856.657767] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c4c0c9ebed0b4cee88365f4027f32148 in queue reply_156db6c4256a4c58884afce15e6f5340
[  856.663896] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c4c0c9ebed0b4cee88365f4027f32148
[  856.958575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.958773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.958897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.959159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.959297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.959544] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.959694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.959818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.960048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.960294] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.960443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.960557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.960757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.960893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.961035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.961283] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.961427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.961541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.961744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.961996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  856.962242] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  856.962402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  856.962520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  856.962729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  856.962884] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  856.963117] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6a0b1031d9b349d49fce7cacfd124e31 in queue reply_156db6c4256a4c58884afce15e6f5340
[  856.973643] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6a0b1031d9b349d49fce7cacfd124e31
[  857.003124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  857.004144] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  857.004144] env[60890]: ERROR oslo_messaging.rpc.server 
[  857.159820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  857.160323] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  857.160323] env[60890]: ERROR oslo_messaging.rpc.server 
[  857.465767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  857.466440] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  857.466440] env[60890]: ERROR oslo_messaging.rpc.server 
[  857.540375] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.540696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.540796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.541079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.541277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.541554] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.541721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.541841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.542065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.542310] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.542465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.542585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.542794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.542937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.543087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.543335] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.543493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.543615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.543825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.543963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.544219] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.544377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.544498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.544766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.544863] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  857.545100] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bc70849334554d4abb9edcf84decb91d in queue reply_156db6c4256a4c58884afce15e6f5340
[  857.550063] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bc70849334554d4abb9edcf84decb91d
[  857.715519] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.716219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.716219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.716317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.716422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.716696] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.716860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.716983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.717209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.717460] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.717619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.717738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.717949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.718092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.718241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.718496] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.718653] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.718775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.719070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.719245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  857.719494] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  857.719655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  857.719777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  857.719992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  857.720146] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  857.720378] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d7ccc2b3810644b39f607d5805e0c8ea in queue reply_156db6c4256a4c58884afce15e6f5340
[  857.726625] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d7ccc2b3810644b39f607d5805e0c8ea
[  858.002391] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.002603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.002752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.003160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.003422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.003701] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.003862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.003987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.004231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.004511] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.004675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.004844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.005052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.005199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.005350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.005605] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.005787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.005941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.006152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.006291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.006533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.006690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.006810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.007020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.007166] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  858.007433] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9bec1c9596504c9984e3dcc4f1816da4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  858.013608] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9bec1c9596504c9984e3dcc4f1816da4
[  858.049658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  858.050116] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  858.050116] env[60890]: ERROR oslo_messaging.rpc.server 
[  858.222721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  858.223239] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  858.223239] env[60890]: ERROR oslo_messaging.rpc.server 
[  858.509328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  858.509856] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  858.509856] env[60890]: ERROR oslo_messaging.rpc.server 
[  858.585705] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.586051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.586090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.586326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.586477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.586734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.586892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.587013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.587231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.587475] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.587629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.587749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.587956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.588113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.588265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.588516] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.588671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.588800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.589106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.589248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.589488] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.589716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.589879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.590112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.590251] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  858.590485] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d298e04ee2b244b6a589c1989d8f46ba in queue reply_156db6c4256a4c58884afce15e6f5340
[  858.595480] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d298e04ee2b244b6a589c1989d8f46ba
[  858.827418] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.827614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.827737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.828032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.828182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.828453] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.828585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.828698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.829000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.829260] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.829413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.829526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.829735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.829870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.830014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.830258] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.830409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.830522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.830744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.830851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  858.831128] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  858.831235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  858.831342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  858.831552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  858.831687] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('49abf6ce-424c-4bec-b6ec-b30cfcc1f780',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  858.831917] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 30f39d79114e4c37ba19512ccdb1a148 in queue reply_156db6c4256a4c58884afce15e6f5340
[  858.836654] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 30f39d79114e4c37ba19512ccdb1a148
[  859.045112] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.045316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.045444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.045714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.045894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.046173] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.046334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.046456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.046678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.046924] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.047082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.047384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.047620] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.047778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.047937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.048232] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.048397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.048520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.048731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.048872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.049115] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.049272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.049391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.049600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.049737] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  859.050007] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 34f7d13723e1480fadc206a156504698 in queue reply_156db6c4256a4c58884afce15e6f5340
[  859.055080] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 34f7d13723e1480fadc206a156504698
[  859.092252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  859.092684] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  859.092684] env[60890]: ERROR oslo_messaging.rpc.server 
[  859.334173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-49abf6ce-424c-4bec-b6ec-b30cfcc1f780" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  859.334686] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  859.334686] env[60890]: ERROR oslo_messaging.rpc.server 
[  859.552896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  859.553417] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  859.553417] env[60890]: ERROR oslo_messaging.rpc.server 
[  859.639454] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.639730] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.639827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.640145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.640319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.640597] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.640766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.640891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.641134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.641413] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.641584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.641709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.641954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.642100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.642267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.642560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.642842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.643009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.643232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.643374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  859.643654] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  859.643841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  859.643975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  859.644221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  859.644380] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  859.644627] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 43d8a302b7ee410a9237b966b1ba8aa5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  859.650136] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 43d8a302b7ee410a9237b966b1ba8aa5
[  860.092209] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.092428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.092556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.092854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.093012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.093273] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.093435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.093556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.094114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.094390] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.094554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.094677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.094924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.095075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.095228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.095489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.095648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.095773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.095986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.096216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.096469] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.096629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.096753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.096979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.097140] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  860.097391] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0327662b31ea48c9b02d2d583aa1555d in queue reply_156db6c4256a4c58884afce15e6f5340
[  860.102426] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0327662b31ea48c9b02d2d583aa1555d
[  860.146738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  860.147398] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  860.147398] env[60890]: ERROR oslo_messaging.rpc.server 
[  860.527414] env[60890]: INFO networking_nsxv3.common.synchronization [req-c741c015-96e7-4d6b-9b66-799c654a1eb3 req-650dacdf-b6e0-4350-a408-1485395528b4 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  860.527984] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  860.528478] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.528478] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.528557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.528720] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.528810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.528971] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.529070] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.529131] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.529285] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.529423] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.529511] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.529569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.530467] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.530467] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.530467] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  860.530467] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e7ff8bd4c9aa4fd3993b2b9f1a266c54 in queue reply_156db6c4256a4c58884afce15e6f5340
[  860.559585] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e7ff8bd4c9aa4fd3993b2b9f1a266c54
[  860.599717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  860.600245] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  860.600245] env[60890]: ERROR oslo_messaging.rpc.server 
[  860.703100] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.703437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.703629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.704186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.704474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.704958] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.705216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.705438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.705802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.706214] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.706433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.706565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.706787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.706938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.707091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.707349] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.709882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.709882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.709882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.709882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  860.709882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  860.709882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  860.709882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  860.709882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  860.709882] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  860.709882] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fbbf7f7df0224a379cb931392f981006 in queue reply_156db6c4256a4c58884afce15e6f5340
[  860.713864] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fbbf7f7df0224a379cb931392f981006
[  861.032919] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.033085] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.033150] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.033368] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.033473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.033657] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.033755] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.033815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.033961] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.034070] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.034132] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  861.034189] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.146992] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.146992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.146992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.146992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.146992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.147291] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.147407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.147556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.147748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.148036] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.148188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.148310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.148530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.148666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.148819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.149081] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.149239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.149505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.149566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.149702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.149949] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.150102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.150222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.150437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.150574] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  861.150801] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5a3a2639f7a24d809bc831a35ed513ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  861.157685] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5a3a2639f7a24d809bc831a35ed513ad
[  861.210984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  861.211505] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  861.211505] env[60890]: ERROR oslo_messaging.rpc.server 
[  861.653565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  861.653565] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  861.653565] env[60890]: ERROR oslo_messaging.rpc.server 
[  861.751149] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.751449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.751561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.751802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.751963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.752252] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-285 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.752416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.752536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.752761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.753022] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.753191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.753312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.753522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.753668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.753816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-285" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.754065] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.754220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.754377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.754598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.754735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  861.755005] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  861.755162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  861.755281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  861.755489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  861.755631] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fac3896e-65c0-4b9b-95a4-13e508fcdeaf',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  861.755903] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1bdabc4eb47c45bca67b7c4310533018 in queue reply_156db6c4256a4c58884afce15e6f5340
[  861.762116] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1bdabc4eb47c45bca67b7c4310533018
[  862.188267] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  862.188462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  862.188590] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  862.188862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  862.189017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  862.189276] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  862.189430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  862.189543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  862.189755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  862.189998] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  862.190153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  862.190270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  862.190476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  862.190618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  862.190810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  862.191014] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  862.191169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  862.191288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  862.191496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  862.191631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  862.192081] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  862.192252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  862.192377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  862.192589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  862.192751] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  862.192988] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg af4601f6dd1d4bdd8ed4dc6f5685e9b6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  862.201220] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg af4601f6dd1d4bdd8ed4dc6f5685e9b6
[  862.258257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-fac3896e-65c0-4b9b-95a4-13e508fcdeaf" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  862.258870] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  862.258870] env[60890]: ERROR oslo_messaging.rpc.server 
[  862.695341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  862.695899] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  862.695899] env[60890]: ERROR oslo_messaging.rpc.server 
[  863.050756] env[60890]: INFO networking_nsxv3.common.synchronization [req-5cb1652d-8f2e-4b5a-80ea-297775307349 req-90c8e0bc-378c-490f-b4b9-680a11c2159f tempest-ServersNegativeTestJSON-229127054 tempest-ServersNegativeTestJSON-229127054-project-member] Enqueued Resource ID: 8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d with Priority: HIGHEST for action security_group_members
[  863.051263] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d with Priority: HIGHEST for action security_group_members
[  863.051575] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.051705] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.051774] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.051975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.052084] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.052224] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.052323] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.052385] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.052557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.052702] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.052810] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.052878] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.053036] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.053098] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.053192] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  863.053371] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 134c9a0b1f9b422dade6f3b61852d813 in queue reply_156db6c4256a4c58884afce15e6f5340
[  863.061033] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 134c9a0b1f9b422dade6f3b61852d813
[  863.236326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.236326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.236326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.236326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.236326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.242321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.242321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.242321] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.242321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.242321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.242321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.242321] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  863.242321] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fb9b93544cdd46108e6fea1f86c51117 in queue reply_156db6c4256a4c58884afce15e6f5340
[  863.248539] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fb9b93544cdd46108e6fea1f86c51117
[  863.560523] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.560681] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.560752] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.560966] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.561065] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.561240] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  863.561336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  863.561396] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  863.561544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  863.561617] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.561700] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d already deleted.
[  863.561777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-8c6ebb0b-71ef-4dbc-a4b0-7af11f842a1d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.743521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  863.744029] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  863.744029] env[60890]: ERROR oslo_messaging.rpc.server 
[  864.312563] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  864.312841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  864.312926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  864.313188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  864.313338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  864.313599] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  864.313766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  864.313884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  864.314103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  864.314348] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  864.314506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  864.314627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  864.314925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  864.315089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  864.316032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  864.316032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  864.316032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  864.316032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  864.316032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  864.316212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  864.316363] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  864.316519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  864.316637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  864.316844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  864.316979] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  864.317288] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2d3e359ae29840028c08a72039a6f866 in queue reply_156db6c4256a4c58884afce15e6f5340
[  864.322558] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2d3e359ae29840028c08a72039a6f866
[  864.511548] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 77978700617e4affbdaa98c4180ef79f in queue reply_156db6c4256a4c58884afce15e6f5340
[  864.524016] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 77978700617e4affbdaa98c4180ef79f
[  864.819833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  864.820374] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  864.820374] env[60890]: ERROR oslo_messaging.rpc.server 
[  865.426575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  865.426575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  865.426575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  865.426575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  865.427913] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  865.427913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  865.427913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  865.427913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  865.427913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  865.427913] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53405196-533f-4703-94af-8c56f6a70d62 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  865.427913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  865.427913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  865.428385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  865.428385] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53405196-533f-4703-94af-8c56f6a70d62',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  865.428385] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 121324e4f90a4ea28336cb128a0f805f in queue reply_156db6c4256a4c58884afce15e6f5340
[  865.433169] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 121324e4f90a4ea28336cb128a0f805f
[  865.931427] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53405196-533f-4703-94af-8c56f6a70d62" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  865.931427] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  865.931427] env[60890]: ERROR oslo_messaging.rpc.server 
[  866.247155] env[60890]: INFO networking_nsxv3.common.synchronization [None req-4d88b7c6-1855-4fbc-819c-36a27781df14 None None] Enqueued Resource ID: 69f74e34-7487-4b97-aef6-97c01d6a129a with Priority: HIGHEST for action port
[  866.247155] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 69f74e34-7487-4b97-aef6-97c01d6a129a with Priority: HIGHEST for action port
[  866.247155] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  866.247155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  866.247155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  866.247155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  866.247155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  866.247155] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-69f74e34-7487-4b97-aef6-97c01d6a129a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  866.247155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-69f74e34-7487-4b97-aef6-97c01d6a129a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  866.247155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-69f74e34-7487-4b97-aef6-97c01d6a129a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  866.247155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-69f74e34-7487-4b97-aef6-97c01d6a129a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  866.247155] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('69f74e34-7487-4b97-aef6-97c01d6a129a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  866.247155] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 52598b87af8347b7a9db66faba1ffb84 in queue reply_156db6c4256a4c58884afce15e6f5340
[  866.251428] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 52598b87af8347b7a9db66faba1ffb84
[  866.748373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-69f74e34-7487-4b97-aef6-97c01d6a129a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  866.748630] env[60890]: Traceback (most recent call last):
[  866.748665] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  866.748665] env[60890]:     timer()
[  866.748665] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  866.748665] env[60890]:     cb(*args, **kw)
[  866.748665] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  866.748665] env[60890]:     result = function(*args, **kwargs)
[  866.748665] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  866.748665] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  866.748665] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  866.748665] env[60890]:     return method(*args, **kwargs)
[  866.748665] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  866.748665] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  866.748665] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  866.748665] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  866.748665] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  866.748665] env[60890]:     result = self.transport._send(
[  866.749061] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  866.749061] env[60890]:     return self._driver.send(target, ctxt, message,
[  866.749061] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  866.749061] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  866.749061] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  866.749061] env[60890]:     raise result
[  866.749061] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  866.749061] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  866.830993] env[60890]: INFO networking_nsxv3.common.synchronization [req-c0cafa2e-fafd-4848-9979-ccd2e8070f13 req-2740ad17-59f3-4ecc-97cf-efbe76f01d77 tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb with Priority: HIGHEST for action security_group_members
[  866.831532] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb with Priority: HIGHEST for action security_group_members
[  866.831852] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  866.831976] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  866.832064] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  866.832263] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  866.832348] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  866.832605] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-beabdba8-4b41-4b8f-a694-d75a0cc077bb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  866.832605] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  866.832771] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  866.832831] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  866.833012] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  866.833108] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  866.833169] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  866.833343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  866.833442] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  866.833561] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('beabdba8-4b41-4b8f-a694-d75a0cc077bb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  866.833722] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 27a0cd25796f4d0ca3bc85fd7fb25e6e in queue reply_156db6c4256a4c58884afce15e6f5340
[  866.847048] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 27a0cd25796f4d0ca3bc85fd7fb25e6e
[  867.336583] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  867.336583] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  867.336741] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  867.337521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  867.337521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  867.337521] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  867.337521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  867.337521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  867.337521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  867.337521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  867.337804] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: beabdba8-4b41-4b8f-a694-d75a0cc077bb already deleted.
[  867.337804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-beabdba8-4b41-4b8f-a694-d75a0cc077bb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  868.083361] env[60890]: INFO networking_nsxv3.common.synchronization [None req-b777b5cb-03d8-404e-905b-3f6684e043ac None None] Enqueued Resource ID: 5747e7c7-3e68-4e0f-bbfc-3a416456be20 with Priority: HIGHEST for action port
[  868.083643] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5747e7c7-3e68-4e0f-bbfc-3a416456be20 with Priority: HIGHEST for action port
[  868.084221] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  868.084346] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  868.084411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  868.084612] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  868.084699] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  868.084867] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-5747e7c7-3e68-4e0f-bbfc-3a416456be20 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  868.084969] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-5747e7c7-3e68-4e0f-bbfc-3a416456be20" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  868.085022] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-5747e7c7-3e68-4e0f-bbfc-3a416456be20" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  868.085224] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-5747e7c7-3e68-4e0f-bbfc-3a416456be20" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  868.085303] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5747e7c7-3e68-4e0f-bbfc-3a416456be20',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  868.085478] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 64385a20c1c84548a60f18f41f4a50b8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  868.092387] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 64385a20c1c84548a60f18f41f4a50b8
[  868.588038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-5747e7c7-3e68-4e0f-bbfc-3a416456be20" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  868.588134] env[60890]: Traceback (most recent call last):
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  868.588176] env[60890]:     timer()
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  868.588176] env[60890]:     cb(*args, **kw)
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  868.588176] env[60890]:     result = function(*args, **kwargs)
[  868.588176] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  868.588176] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  868.588176] env[60890]:     return method(*args, **kwargs)
[  868.588176] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  868.588176] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  868.588176] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  868.588176] env[60890]:     result = self.transport._send(
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  868.588176] env[60890]:     return self._driver.send(target, ctxt, message,
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  868.588176] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  868.588176] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  868.588176] env[60890]:     raise result
[  868.588658] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  868.588658] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  873.084712] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  873.084712] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  875.940973] env[60890]: INFO networking_nsxv3.common.synchronization [req-5f33d22c-2a92-4771-a90b-70bee9db8bf7 req-3ad8abff-4ba2-4464-864a-2b69b875b1bf tempest-ImagesTestJSON-598093153 tempest-ImagesTestJSON-598093153-project-member] Enqueued Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  875.941505] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  875.941817] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  875.941938] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  875.942003] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  875.942194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  875.942277] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  875.942421] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f0a2229c-87aa-4f25-9157-b1b2e986b23e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  875.942514] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  875.942574] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  875.942739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  875.942878] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  875.942967] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  875.943024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  875.943163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  875.943237] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  875.943320] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f0a2229c-87aa-4f25-9157-b1b2e986b23e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  875.944609] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 31b638dba1b941d7829412490998eb17 in queue reply_156db6c4256a4c58884afce15e6f5340
[  875.978635] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 31b638dba1b941d7829412490998eb17
[  876.232011] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.232011] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  876.232011] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  876.232011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.232557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.232557] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  876.232557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  876.232662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  876.232947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  876.233015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.233252] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  876.233406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  876.233522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  876.233788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  876.233963] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  876.234733] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1f8888bde44c49fca2d85e4e8d2f146b in queue reply_156db6c4256a4c58884afce15e6f5340
[  876.240110] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1f8888bde44c49fca2d85e4e8d2f146b
[  876.446650] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  876.446773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  876.446844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  876.447055] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  876.447167] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.447363] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  876.447466] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  876.447525] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  876.447675] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  876.447752] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.447836] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e already deleted.
[  876.447942] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.737910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  876.737910] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  876.737910] env[60890]: ERROR oslo_messaging.rpc.server 
[  877.325600] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  877.325857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  877.326021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  877.326299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  877.326448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  877.326704] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  877.326862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  877.326983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  877.327200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  877.327443] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  877.327602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  877.327721] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  877.328009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  877.328160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  877.328305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  877.328553] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  877.328709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  877.328826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  877.329037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  877.329175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  877.329405] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  877.329556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  877.329674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  877.329882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  877.330044] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  877.330249] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 08476ef7cd8040108ab781d618c167c5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  877.334866] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 08476ef7cd8040108ab781d618c167c5
[  877.832452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  877.833021] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  877.833021] env[60890]: ERROR oslo_messaging.rpc.server 
[  878.205818] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.206056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.206194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.206465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.206669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.206905] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.207015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.207155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.207387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.207636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.207826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.207906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.208127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.208283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.208425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.208673] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.208830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.208950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.209200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.209333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.209535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.209685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.209804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.210011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.210145] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  878.210370] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4d151687a8244249b06390ff67bf9437 in queue reply_156db6c4256a4c58884afce15e6f5340
[  878.215962] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4d151687a8244249b06390ff67bf9437
[  878.322850] env[60890]: INFO networking_nsxv3.common.synchronization [req-58e64277-a4e2-45eb-b5ba-bee7cbce34f9 req-71df0363-f897-44e3-896e-9835abb597b4 tempest-InstanceActionsV221TestJSON-1053823549 tempest-InstanceActionsV221TestJSON-1053823549-project-member] Enqueued Resource ID: f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13 with Priority: HIGHEST for action security_group_members
[  878.323347] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13 with Priority: HIGHEST for action security_group_members
[  878.323666] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.323790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.323858] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.324084] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.324175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.324352] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.324453] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.324517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.324712] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.324889] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.325024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.325090] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.325236] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.325314] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.325401] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  878.325576] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 854d498ef1ab461d899579053fe68bdf in queue reply_156db6c4256a4c58884afce15e6f5340
[  878.336533] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 854d498ef1ab461d899579053fe68bdf
[  878.606353] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.606353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.606353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.606353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.606353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.606353] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.606353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.606353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.606747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.606747] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.606862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.606982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.607189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.607329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.607502] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.607730] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.607888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.608016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.608239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.608377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.608618] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.608775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.608895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.609141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.609279] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  878.609510] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0f9b06a3127e454189d32f0cc0209d74 in queue reply_156db6c4256a4c58884afce15e6f5340
[  878.614630] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0f9b06a3127e454189d32f0cc0209d74
[  878.712224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  878.712771] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  878.712771] env[60890]: ERROR oslo_messaging.rpc.server 
[  878.767728] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.767925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.768076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.768346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.768495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.769118] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.769118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.769118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.769236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.769486] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.769641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.769761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.770039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.770122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.770267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.770513] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.770675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.770781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.770990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.771114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.771353] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.771542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.771612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.771818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.771949] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  878.772194] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0e714fe273a24b1e979a0133a90a3771 in queue reply_156db6c4256a4c58884afce15e6f5340
[  878.777632] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0e714fe273a24b1e979a0133a90a3771
[  878.826983] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.827120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.827187] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.827405] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.827497] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.827684] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  878.827783] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  878.827843] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  878.827993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  878.828086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  878.828171] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13 already deleted.
[  878.828250] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.111633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  879.112162] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  879.112162] env[60890]: ERROR oslo_messaging.rpc.server 
[  879.274882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  879.275402] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  879.275402] env[60890]: ERROR oslo_messaging.rpc.server 
[  879.657162] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.657491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.657619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.657980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.658152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.658533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.658771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.658934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.659199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.659475] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.659639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.659794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.660070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.660225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.660385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.660692] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.660864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.660993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.661278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.661429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.661747] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.661920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.662048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.662264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.662438] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  879.662788] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bdde32daaf03404bb84e332d528d3db0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  879.669156] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bdde32daaf03404bb84e332d528d3db0
[  879.812990] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.813211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.813344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.813615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.813762] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.814017] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.814177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.814297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.814514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.814757] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.814941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.815065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.815272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.815412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.815598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.815811] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.815979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.816178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.816394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.816534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  879.816787] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  879.816929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  879.817048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  879.817255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  879.817391] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  879.817610] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 59c754c60feb4c84a2ffdd67cb2d395e in queue reply_156db6c4256a4c58884afce15e6f5340
[  879.822343] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 59c754c60feb4c84a2ffdd67cb2d395e
[  880.166071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  880.166616] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  880.166616] env[60890]: ERROR oslo_messaging.rpc.server 
[  880.322363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  880.322980] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  880.322980] env[60890]: ERROR oslo_messaging.rpc.server 
[  880.700689] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.700952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.701149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.701485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.701706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.702058] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.702296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.702475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.702767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.703075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.703338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.703520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.703787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.704014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.704234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.704592] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.704856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.705042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.705315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.705504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.705811] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.706062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.706232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.706502] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.706702] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  880.706988] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0a0320f4443a48e5b007c56dabb9474d in queue reply_156db6c4256a4c58884afce15e6f5340
[  880.713440] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0a0320f4443a48e5b007c56dabb9474d
[  880.860036] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.860380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.860551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.860850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.861043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.861336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.861543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.861694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.861949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.862232] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.862428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.862578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.862850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.863032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.863219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.863503] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.863702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.863853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.864129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.864298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  880.864580] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  880.864772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  880.864948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  880.865194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  880.865360] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  880.865615] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 775edf224b4c4656806e7c7e543624a7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  880.870665] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 775edf224b4c4656806e7c7e543624a7
[  881.209834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  881.210429] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  881.210429] env[60890]: ERROR oslo_messaging.rpc.server 
[  881.367314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  881.367818] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  881.367818] env[60890]: ERROR oslo_messaging.rpc.server 
[  881.747202] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.747556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.747556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.747823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.747979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.748264] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.748436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.748556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.748776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.749024] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.749185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.749303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.749514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.749657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.749806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.750062] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.750215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.750334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.750543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.750678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.750921] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.751077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.751194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.751403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.751536] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  881.751761] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 99260e890ba24cf1b79baeef6bb9bb76 in queue reply_156db6c4256a4c58884afce15e6f5340
[  881.757491] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 99260e890ba24cf1b79baeef6bb9bb76
[  881.906130] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.906350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.906481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.906747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.906896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.907184] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.907343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.907464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.907679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.907922] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.908099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.908221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.908430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.908573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.908723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.908973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.909130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.909249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.909456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.909592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  881.909833] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  881.909993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  881.910110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  881.910314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  881.910449] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  881.910676] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6e7c7531e0b24bf7b0454ab239770c09 in queue reply_156db6c4256a4c58884afce15e6f5340
[  881.916658] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6e7c7531e0b24bf7b0454ab239770c09
[  882.254318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  882.254909] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  882.254909] env[60890]: ERROR oslo_messaging.rpc.server 
[  882.412645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  882.413178] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  882.413178] env[60890]: ERROR oslo_messaging.rpc.server 
[  882.792430] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.792729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.792836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.793137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.793287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.793547] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.793708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.793830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.794051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.794300] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.794456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.794576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.794823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.794982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.795136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.795386] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.795629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.795763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.796017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.796162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.796405] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.796562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.796681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.796890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.797028] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  882.797268] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6ded0b673a754661b71a6dc7ed1ee54a in queue reply_156db6c4256a4c58884afce15e6f5340
[  882.802442] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6ded0b673a754661b71a6dc7ed1ee54a
[  882.953741] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.955401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.955401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.955401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.955401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.955401] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.955401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.955401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.955401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.955717] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.955999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.955999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.956076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.956212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.956365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.956680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.956837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.956958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.957204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.957345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  882.957591] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  882.957746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  882.957865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  882.958077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  882.958208] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  882.958436] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ae4783ad9be7406e8bb17bb6e1fea6c8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  882.963637] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ae4783ad9be7406e8bb17bb6e1fea6c8
[  883.300145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  883.301426] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  883.301426] env[60890]: ERROR oslo_messaging.rpc.server 
[  883.324684] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  883.324787] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  883.461002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  883.461570] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  883.461570] env[60890]: ERROR oslo_messaging.rpc.server 
[  883.837962] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  883.837962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  883.837962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  883.838366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  883.838450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  883.838725] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  883.838919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  883.839054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  883.839330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  883.839529] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  883.839689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  883.839811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  883.840204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  883.840357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  883.840512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  883.841301] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  883.841301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  883.841301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  883.841301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  883.841471] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  883.841703] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  883.841916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  883.842058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  883.842340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  883.842524] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  883.842783] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 68a45ce383b94c449ceb19fbd00ad85c in queue reply_156db6c4256a4c58884afce15e6f5340
[  883.848027] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 68a45ce383b94c449ceb19fbd00ad85c
[  884.004026] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.004026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.004026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.004026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.004026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.004026] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.004294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.004324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.004540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.004814] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.005007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.005135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.005348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.005493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.005645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.005900] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.006060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.006183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.006391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.006529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.006771] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.006929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.007051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.007343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.007488] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  884.007724] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 444baae63f7047f7b0a4501d5e2fe5eb in queue reply_156db6c4256a4c58884afce15e6f5340
[  884.012546] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 444baae63f7047f7b0a4501d5e2fe5eb
[  884.345281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  884.345779] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  884.345779] env[60890]: ERROR oslo_messaging.rpc.server 
[  884.510184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  884.511162] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  884.511162] env[60890]: ERROR oslo_messaging.rpc.server 
[  884.892464] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.892464] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.892464] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.892464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.893196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.893196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.893196] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.893196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.893362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.893426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.893561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  884.893798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  884.893960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  884.894079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  884.894283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  884.894416] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  884.894645] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 468dffd2138345f0a0cb277c47e1b249 in queue reply_156db6c4256a4c58884afce15e6f5340
[  884.899911] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 468dffd2138345f0a0cb277c47e1b249
[  885.052977] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.053193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.053321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.053585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.053779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.054053] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.054220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.054341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.054580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.054872] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.055052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.055174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.055382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.055524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.055672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.055919] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.056088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.056207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.056413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.056547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.056783] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-513390d9-ee31-4f74-ab3d-d7552019e84d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.056936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.057058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.057530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.057700] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('513390d9-ee31-4f74-ab3d-d7552019e84d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  885.057952] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d46ef1e585b94452b1d1ccb9dcb36c10 in queue reply_156db6c4256a4c58884afce15e6f5340
[  885.063490] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d46ef1e585b94452b1d1ccb9dcb36c10
[  885.397041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  885.397573] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  885.397573] env[60890]: ERROR oslo_messaging.rpc.server 
[  885.521312] env[60890]: INFO networking_nsxv3.common.synchronization [req-ed1b67ca-2b9b-4908-9ed5-413f9130d95f req-c7bc953c-beca-4dc6-9d4e-20aa8b23989e tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 with Priority: HIGHEST for action security_group_members
[  885.521845] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 with Priority: HIGHEST for action security_group_members
[  885.522163] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.522305] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.522394] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.522553] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.522638] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.522820] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b9ba6520-46ed-4da6-847f-1a8360a3f912 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.522931] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.522986] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.523153] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.523290] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.523381] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.523440] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.523574] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.523651] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.523737] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b9ba6520-46ed-4da6-847f-1a8360a3f912',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  885.523910] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 11b54be23ffe437dbd0c4b24ba91ee4a in queue reply_156db6c4256a4c58884afce15e6f5340
[  885.533597] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 11b54be23ffe437dbd0c4b24ba91ee4a
[  885.560539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-513390d9-ee31-4f74-ab3d-d7552019e84d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  885.561034] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  885.561034] env[60890]: ERROR oslo_messaging.rpc.server 
[  885.942728] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.943180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.943180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.943332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.943503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.943699] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.943856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.943978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.944216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.944468] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.944625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.944740] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.944987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.945137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.945330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.945594] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.945751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.945872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.946112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.946264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  885.946508] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  885.946887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  885.947038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  885.947254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  885.947393] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  885.947625] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0e823a49f2464c099249d0892b5b20e6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  885.953631] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0e823a49f2464c099249d0892b5b20e6
[  886.026231] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.026520] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.026520] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.026649] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.026728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.026926] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.027026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.027078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.027235] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.027312] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.027397] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 already deleted.
[  886.027473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.361008] env[60890]: INFO networking_nsxv3.common.synchronization [req-5f33d22c-2a92-4771-a90b-70bee9db8bf7 req-3251887a-1c43-4973-90fc-7771c24dd1db tempest-ImagesTestJSON-598093153 tempest-ImagesTestJSON-598093153-project-member] Enqueued Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  886.361522] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e with Priority: HIGHEST for action security_group_members
[  886.361834] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.361956] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.362020] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.362222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.362322] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.362475] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f0a2229c-87aa-4f25-9157-b1b2e986b23e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.362569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.362627] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.362824] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.362946] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.363040] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.363095] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.363232] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.363309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.363395] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f0a2229c-87aa-4f25-9157-b1b2e986b23e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  886.363599] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a1a159c6a6a04ad28ab680b5b49cca74 in queue reply_156db6c4256a4c58884afce15e6f5340
[  886.371231] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a1a159c6a6a04ad28ab680b5b49cca74
[  886.423581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.423581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.423581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.423581] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.423581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.424534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.424845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.425241] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.425574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.425842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.426235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.426499] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  886.426855] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6b5a37222b0b4218848732de7287c6db in queue reply_156db6c4256a4c58884afce15e6f5340
[  886.436894] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6b5a37222b0b4218848732de7287c6db
[  886.450241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  886.450628] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  886.450628] env[60890]: ERROR oslo_messaging.rpc.server 
[  886.865254] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.866645] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.866645] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f0a2229c-87aa-4f25-9157-b1b2e986b23e already deleted.
[  886.866645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f0a2229c-87aa-4f25-9157-b1b2e986b23e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.929791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  886.930274] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  886.930274] env[60890]: ERROR oslo_messaging.rpc.server 
[  887.123062] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.123353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.123353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.123552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.123699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.123954] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.124130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.124254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.124473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.124722] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.124920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.125047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.125259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.125403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.125553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.125868] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.126036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.126160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.126372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.126509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.126750] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.126908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.127031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.127239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.127377] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  887.127610] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 39959c6a27cf439cb1748fff88fd786e in queue reply_156db6c4256a4c58884afce15e6f5340
[  887.132505] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 39959c6a27cf439cb1748fff88fd786e
[  887.381494] env[60890]: INFO networking_nsxv3.common.synchronization [None req-4fcd4fd2-eb90-4534-b58a-4896eb27acf8 tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 with Priority: HIGHEST for action security_group_rules
[  887.382001] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 with Priority: HIGHEST for action security_group_rules
[  887.382317] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.382439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.382507] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.382720] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.382822] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.382974] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-498c7d18-56d5-4041-ba4b-f3cb998fea54 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.383072] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.383134] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.383336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.383416] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('498c7d18-56d5-4041-ba4b-f3cb998fea54',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  887.383590] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ab5dd6ae53f94cca89a2bd7b289c33f8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  887.396372] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ab5dd6ae53f94cca89a2bd7b289c33f8
[  887.469561] env[60890]: INFO networking_nsxv3.common.synchronization [req-6b5a274b-c1b9-4876-b12f-d6e096c8c8ee req-41bf0398-bbe5-4917-b2d3-0581af314f4f tempest-ServerRescueTestJSON-523553051 tempest-ServerRescueTestJSON-523553051-project-member] Enqueued Resource ID: ab97351c-3e27-4028-9501-5c8c2ed8e69b with Priority: HIGHEST for action security_group_members
[  887.470032] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ab97351c-3e27-4028-9501-5c8c2ed8e69b with Priority: HIGHEST for action security_group_members
[  887.470343] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.470465] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.470529] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.470728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.470815] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.470963] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ab97351c-3e27-4028-9501-5c8c2ed8e69b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.471055] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ab97351c-3e27-4028-9501-5c8c2ed8e69b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.471114] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ab97351c-3e27-4028-9501-5c8c2ed8e69b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.471374] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ab97351c-3e27-4028-9501-5c8c2ed8e69b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.471519] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.471611] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.471670] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.473228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.473350] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.473449] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ab97351c-3e27-4028-9501-5c8c2ed8e69b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  887.473626] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ba1582a3b87943bd9c6e4db5a58a210c in queue reply_156db6c4256a4c58884afce15e6f5340
[  887.486621] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ba1582a3b87943bd9c6e4db5a58a210c
[  887.531221] env[60890]: INFO networking_nsxv3.common.synchronization [None req-0733e368-3613-4221-844a-af994ffb876b tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 with Priority: HIGHEST for action security_group_rules
[  887.531685] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 with Priority: HIGHEST for action security_group_rules
[  887.532054] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.532180] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.532250] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.532448] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.532534] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.532684] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-498c7d18-56d5-4041-ba4b-f3cb998fea54 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.532762] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.629641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  887.630173] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  887.630173] env[60890]: ERROR oslo_messaging.rpc.server 
[  887.657933] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.658137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.658266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.658534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.658684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.658942] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.659114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.659238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.659457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.659708] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.659868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.659991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.660239] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.660388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.660541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.660798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.660958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.661079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.661291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.661429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.661675] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.661833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.661955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.662165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.662300] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  887.662571] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cf9dd3c49c1048848f6ce65b8f4942de in queue reply_156db6c4256a4c58884afce15e6f5340
[  887.667964] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cf9dd3c49c1048848f6ce65b8f4942de
[  887.904323] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.904470] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.904541] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.904874] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.905048] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.905158] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 already deleted.
[  887.905251] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.905395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.905566] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.905647] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('498c7d18-56d5-4041-ba4b-f3cb998fea54',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  887.905863] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 761a9c998aec436f9021979a18aee97c in queue reply_156db6c4256a4c58884afce15e6f5340
[  887.917920] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 761a9c998aec436f9021979a18aee97c
[  887.928918] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.929121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.929253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.929524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.929674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.929936] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.930099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.930224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.930505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.930756] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.930935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.931070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.931287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.931442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.931577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.931830] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.931986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.932116] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.932325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.932458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.932702] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.933254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.933413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.933673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.933814] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  887.934067] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f935343f81204573bb93ec35e6901c1a in queue reply_156db6c4256a4c58884afce15e6f5340
[  887.939078] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f935343f81204573bb93ec35e6901c1a
[  887.977981] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.978160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.978234] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.978444] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.978536] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.978721] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  887.978820] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  887.978882] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  887.979027] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  887.979104] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  887.979186] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ab97351c-3e27-4028-9501-5c8c2ed8e69b already deleted.
[  887.979263] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ab97351c-3e27-4028-9501-5c8c2ed8e69b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.165863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  888.166392] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  888.166392] env[60890]: ERROR oslo_messaging.rpc.server 
[  888.245465] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.245810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.245890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.246148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.246304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.246596] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-656 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.246736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.246891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.247045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.247314] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.247478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.247599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.247808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.247951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.248130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-656" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.248380] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.248564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.248697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.248907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.249044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.249283] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-e21327e9-0502-4af0-84ed-34722dba7c32 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.249466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.249545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.249748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.249914] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('e21327e9-0502-4af0-84ed-34722dba7c32',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  888.250195] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9614f2a2ca0548bfbde7b272d5b21576 in queue reply_156db6c4256a4c58884afce15e6f5340
[  888.260483] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9614f2a2ca0548bfbde7b272d5b21576
[  888.424686] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.424863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.424952] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.425142] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.425234] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.425329] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 already deleted.
[  888.425411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.436082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  888.436553] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  888.436553] env[60890]: ERROR oslo_messaging.rpc.server 
[  888.753516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-e21327e9-0502-4af0-84ed-34722dba7c32" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  888.753516] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  888.753516] env[60890]: ERROR oslo_messaging.rpc.server 
[  888.759972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.760200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.760332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.760598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.760746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.761007] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.761169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.761292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.761512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.761760] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.761917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.762038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.762257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.762403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.762553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.762806] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.762965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.763084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.763294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.763431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.763670] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.763826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.763946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.764182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.764325] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  888.764553] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1e5504bbbb75471ca94659b0a834625f in queue reply_156db6c4256a4c58884afce15e6f5340
[  888.769878] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1e5504bbbb75471ca94659b0a834625f
[  888.984374] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.984570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.984702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.985056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.985207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.985469] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.985631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.985755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.986004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.986253] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.986409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.986528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.986735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.986876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.987028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.987320] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.987484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.987606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.987818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.987960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  888.988232] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  888.988390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  888.988512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  888.988813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  888.988955] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  888.989185] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9b804267930c419b8b2118893b5fe962 in queue reply_156db6c4256a4c58884afce15e6f5340
[  888.994335] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9b804267930c419b8b2118893b5fe962
[  889.266628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  889.267141] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  889.267141] env[60890]: ERROR oslo_messaging.rpc.server 
[  889.491481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  889.492055] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  889.492055] env[60890]: ERROR oslo_messaging.rpc.server 
[  889.829545] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.829993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.829993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.830189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.830312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.830621] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.830781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.830936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.831162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.831422] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.831579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.831698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.831951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.832124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.832331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.832553] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.832687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.832828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.833047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.833187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.833472] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.833635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.833756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.833968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.834141] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  889.834389] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ce6a905907ee4b23b9b1fb48022563da in queue reply_156db6c4256a4c58884afce15e6f5340
[  889.840696] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ce6a905907ee4b23b9b1fb48022563da
[  889.959468] env[60890]: INFO networking_nsxv3.common.synchronization [req-58e64277-a4e2-45eb-b5ba-bee7cbce34f9 req-d2a9396e-ec91-4c16-95c0-e99d020c5796 tempest-InstanceActionsV221TestJSON-1053823549 tempest-InstanceActionsV221TestJSON-1053823549-project-member] Enqueued Resource ID: f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13 with Priority: HIGHEST for action security_group_members
[  889.960243] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13 with Priority: HIGHEST for action security_group_members
[  889.960596] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.960725] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.960788] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.961007] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.961176] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.961273] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.961373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.961429] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.961587] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.961740] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  889.961893] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  889.961893] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  889.962055] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  889.962139] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  889.962229] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  889.962411] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f443d09a236044e2ab5d01edcb76ad9d in queue reply_156db6c4256a4c58884afce15e6f5340
[  889.976246] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f443d09a236044e2ab5d01edcb76ad9d
[  890.003412] env[60890]: INFO networking_nsxv3.common.synchronization [req-271d85bf-89bc-41bf-a402-4ab455990cb1 req-142e1082-460a-466a-90df-dc13a262774f tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  890.003913] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  890.004292] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.004419] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.004481] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.004681] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.004882] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.004971] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.005674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.005674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.005674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.005674] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.005674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.005674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.005674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.005871] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.005899] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  890.006815] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0ee1d9e0945841049f5a6529f314e65f in queue reply_156db6c4256a4c58884afce15e6f5340
[  890.017847] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0ee1d9e0945841049f5a6529f314e65f
[  890.037165] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.037421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.037549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.038033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.038033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.038244] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.038404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.038578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.038736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.039061] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.039151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.039240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.039448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.039589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.039737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.039986] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.040163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.040343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.040584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.040726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.040972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.041161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.041270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.041455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.041589] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  890.041822] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e754881f4da948e78a3c3c4329a60fc8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  890.047375] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e754881f4da948e78a3c3c4329a60fc8
[  890.268428] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.268774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.268774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.269028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.269179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.269436] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.269594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.269712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.269978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.270251] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.270412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.270531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.270743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.270884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.271033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.271281] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.271436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.271555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.271910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.271910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.272133] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.272289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.272405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.272668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.272829] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  890.273067] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b94682e52bcd4be381c32f20522c10be in queue reply_156db6c4256a4c58884afce15e6f5340
[  890.278189] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b94682e52bcd4be381c32f20522c10be
[  890.337752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  890.338251] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  890.338251] env[60890]: ERROR oslo_messaging.rpc.server 
[  890.464613] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.464849] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.464964] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.465185] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.465279] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.465480] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.465579] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.465642] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.465804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.465907] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.465974] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13 already deleted.
[  890.466051] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-f9e5f88e-b0a2-44ee-80e6-f90c06ff4e13" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.509761] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.509936] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.510018] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.510231] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.510322] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.510515] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.510614] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.510679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.510831] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.510911] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.510997] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  890.511075] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.544959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  890.545393] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  890.545393] env[60890]: ERROR oslo_messaging.rpc.server 
[  890.775736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  890.775835] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  890.775835] env[60890]: ERROR oslo_messaging.rpc.server 
[  890.937266] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.937491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.937621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.937897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.938045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.938360] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.938469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.938583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.938799] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.939366] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.939366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.939366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.939577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.939652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.939786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.940047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.940207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.940325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.940534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.940670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  890.940909] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  890.941065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  890.941185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  890.941391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  890.941530] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  890.941760] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 90efcbf6488c41f4851eb7fb6ea877ad in queue reply_156db6c4256a4c58884afce15e6f5340
[  890.949373] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 90efcbf6488c41f4851eb7fb6ea877ad
[  891.095182] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.095390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.095515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.095787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.095960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.096248] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.096406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.096525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.096741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.096981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.097147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.097268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.097469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.097608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.097754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.098002] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.098158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.098276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.098477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.098608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.098842] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.098997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.099115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.099318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.099448] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  891.099674] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c60bf73da5c140058f62cedafbcb5dbd in queue reply_156db6c4256a4c58884afce15e6f5340
[  891.104129] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c60bf73da5c140058f62cedafbcb5dbd
[  891.316988] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.317290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.317342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.317579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.317726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.317986] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.318147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.318264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.318484] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.318730] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.318884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.319068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.319325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.319468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.319618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.319869] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.320036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.320158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.320364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.320497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.320732] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  891.320886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  891.321004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  891.321208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  891.321340] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  891.321570] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a4fbe02e5f294058b4b03dab71a74b6e in queue reply_156db6c4256a4c58884afce15e6f5340
[  891.326517] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a4fbe02e5f294058b4b03dab71a74b6e
[  891.444797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  891.445380] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  891.445380] env[60890]: ERROR oslo_messaging.rpc.server 
[  891.601559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  891.602056] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  891.602056] env[60890]: ERROR oslo_messaging.rpc.server 
[  891.823323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  891.823833] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  891.823833] env[60890]: ERROR oslo_messaging.rpc.server 
[  892.021820] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.022037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.022209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.022488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.022641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.022990] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.023158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.023281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.023552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.023856] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.024038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.024166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.024383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.024528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.024679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.025001] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.025191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.025314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.025531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.025669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.025930] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.026103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.026228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.026451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.026623] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  892.026856] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7605d3080a4b4ca192e98176acd013e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  892.032800] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7605d3080a4b4ca192e98176acd013e0
[  892.201747] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.201954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.202084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.202426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.202512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.202769] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.202952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.203078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.203336] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.203542] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.203700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.203820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.204047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.204197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.204349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.204599] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.204754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.204926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.205150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.205288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.205528] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.205685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.205821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.206045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.206183] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  892.206411] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9881cc7e054443c19bfeb46ed8bf49bd in queue reply_156db6c4256a4c58884afce15e6f5340
[  892.211193] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9881cc7e054443c19bfeb46ed8bf49bd
[  892.360682] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.360988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.361027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.361272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.361420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.361765] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.361966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.362114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.362334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.362585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.362851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.362996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.363207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.363349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.363505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.363755] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.363906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.364039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.364251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.364388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.364625] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.364779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.364980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.365192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.365330] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  892.365559] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 728593d75ffd4a70848294c5c0461e04 in queue reply_156db6c4256a4c58884afce15e6f5340
[  892.370391] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 728593d75ffd4a70848294c5c0461e04
[  892.529650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  892.530204] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  892.530204] env[60890]: ERROR oslo_messaging.rpc.server 
[  892.708663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  892.709160] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  892.709160] env[60890]: ERROR oslo_messaging.rpc.server 
[  892.779982] env[60890]: INFO networking_nsxv3.common.synchronization [req-4151299f-90bb-4296-be82-f43d0585b9c9 req-b04e6599-494f-4ccc-94cb-4251ea79fa60 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  892.780582] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  892.780879] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.781012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.781082] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.781280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.781581] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.781581] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.781707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.781707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.781801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.781939] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  892.782029] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  892.782089] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  892.782228] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  892.782306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.782393] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  892.782569] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 1bf363f5aaaa4530b393e86f41d56f4d in queue reply_156db6c4256a4c58884afce15e6f5340
[  892.791104] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1bf363f5aaaa4530b393e86f41d56f4d
[  892.867830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  892.868367] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  892.868367] env[60890]: ERROR oslo_messaging.rpc.server 
[  893.051763] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.051968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.052147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.052420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.052562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.052835] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.052993] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.053107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.053319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.053565] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.053717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.053833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.054036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.054175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.054317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.054560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.054711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.054844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.055085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.055218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.055456] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.055607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.055722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.056332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.056516] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  893.056758] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9e86461c637145c49d3c07b02026328a in queue reply_156db6c4256a4c58884afce15e6f5340
[  893.062533] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9e86461c637145c49d3c07b02026328a
[  893.228246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.228461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.228591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.228870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.229212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.229280] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.229437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.229556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.229776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.230024] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.230184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.230303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.230514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.230657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.230808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.231064] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.231220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.231341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.231547] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.231685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.231920] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.232099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.232232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.232503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.232648] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  893.233007] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fea745e1430e45f2a451f3f09a27f96c in queue reply_156db6c4256a4c58884afce15e6f5340
[  893.238778] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fea745e1430e45f2a451f3f09a27f96c
[  893.284747] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.285018] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.285110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.285330] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.285428] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.285633] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.285736] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.285838] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.286005] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.286088] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.286178] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  893.286260] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.330181] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.330392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.330523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.330788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.330937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.331202] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.331363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.331485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.331705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.331956] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.332138] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.332260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.332470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.332616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.332767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.333106] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.333275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.333411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.333624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.333764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.334007] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.334166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.334288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.334497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.334635] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  893.335023] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 952666ebbf114d618358a2f3c4b9af24 in queue reply_156db6c4256a4c58884afce15e6f5340
[  893.340798] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 952666ebbf114d618358a2f3c4b9af24
[  893.416956] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.417212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.417284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.417576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.417733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.418009] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.418166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.418283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.418507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.418794] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.418959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.419075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.419288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.419426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.419572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.419974] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.419974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.420131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.420390] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.420528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.420768] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  893.420930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  893.421043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  893.421248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  893.421384] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  893.421618] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9cd868390df14979bcceee0bde30b0af in queue reply_156db6c4256a4c58884afce15e6f5340
[  893.428566] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9cd868390df14979bcceee0bde30b0af
[  893.559367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  893.559864] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  893.559864] env[60890]: ERROR oslo_messaging.rpc.server 
[  893.735511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  893.735693] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  893.735693] env[60890]: ERROR oslo_messaging.rpc.server 
[  893.837338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  893.837828] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  893.837828] env[60890]: ERROR oslo_messaging.rpc.server 
[  893.925026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  893.925489] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  893.925489] env[60890]: ERROR oslo_messaging.rpc.server 
[  894.096196] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.096411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.096541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.096813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.096965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.097221] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.097377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.097496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.097712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.097976] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.098143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.098260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.098469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.098606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.098751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.099003] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.099160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.099279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.099487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.099625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.099861] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.100032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.100158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.100366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.100501] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  894.100725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4c213213676844fcb37570d0cb4f643c in queue reply_156db6c4256a4c58884afce15e6f5340
[  894.105912] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4c213213676844fcb37570d0cb4f643c
[  894.292937] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.293124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.293257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.293529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.293680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.293946] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.294109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.294231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.294448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.294696] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.294871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.295000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.295212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.295356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.295510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.295762] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.295956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.296090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.296304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.296444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.296685] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.296845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.296965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.297172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.297309] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  894.297538] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 32cfa3a6e0ab40a38d4d0d5874b6ec55 in queue reply_156db6c4256a4c58884afce15e6f5340
[  894.304112] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 32cfa3a6e0ab40a38d4d0d5874b6ec55
[  894.433131] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.433466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.433466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.433718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.433866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.434121] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.434420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.434420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.434716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.434979] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.435086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.435196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.435408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.435551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.435700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.435987] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.436164] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.436591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.436591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.436676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.436854] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.437012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.437131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.437337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.437472] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  894.437705] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c1779909250c46b2bfec2a98fde80371 in queue reply_156db6c4256a4c58884afce15e6f5340
[  894.443108] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c1779909250c46b2bfec2a98fde80371
[  894.461580] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.461857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.462011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.462282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.463197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.463484] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.463649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.463773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.464012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.464267] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.464426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.464546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.464756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.464992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.465158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.465411] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.465569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.465689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.465925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.466092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.466307] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  894.466457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  894.466576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  894.466785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  894.466923] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  894.467161] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5f9c9179a6c34a838f2e62f71ffbbe52 in queue reply_156db6c4256a4c58884afce15e6f5340
[  894.478894] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5f9c9179a6c34a838f2e62f71ffbbe52
[  894.514868] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg bfbb09c91cef4b6c97c22c481c1796eb in queue reply_156db6c4256a4c58884afce15e6f5340
[  894.527074] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bfbb09c91cef4b6c97c22c481c1796eb
[  894.602236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  894.602750] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  894.602750] env[60890]: ERROR oslo_messaging.rpc.server 
[  894.799492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  894.800026] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  894.800026] env[60890]: ERROR oslo_messaging.rpc.server 
[  894.939947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  894.940497] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  894.940497] env[60890]: ERROR oslo_messaging.rpc.server 
[  894.969489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  894.969964] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  894.969964] env[60890]: ERROR oslo_messaging.rpc.server 
[  895.147899] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.148044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.148139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.148551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.148724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.149002] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.149169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.149293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.149522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.149768] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.149925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.150047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.150256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.150395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.150546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.150798] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.151025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.151173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.151462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.151649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.151926] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.152155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.152300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.152531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.152697] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  895.152959] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 27c68ac1d61a4d9ebe5b2ea84639b117 in queue reply_156db6c4256a4c58884afce15e6f5340
[  895.158793] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 27c68ac1d61a4d9ebe5b2ea84639b117
[  895.351155] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.351361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.351490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.352881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.352881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.352881] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.352881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.352881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.352881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.353338] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.353338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.353338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.353724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.353724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.353724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.354392] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.354392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.354392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.354392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.354772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.354772] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.354970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.355070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.355276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.355409] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  895.355636] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9fd48cc591244a97a5e1a45213e93f46 in queue reply_156db6c4256a4c58884afce15e6f5340
[  895.360828] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9fd48cc591244a97a5e1a45213e93f46
[  895.479729] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.480035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.480076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.480349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.480481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.481090] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.481090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.481214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.481363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.481597] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.481749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.481865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.482894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.482894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.482894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.482894] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.482894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.482894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.483151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.483423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.483490] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.483645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.483758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.484078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.484217] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  895.484451] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 64e943b385514593aa9f495603972282 in queue reply_156db6c4256a4c58884afce15e6f5340
[  895.490549] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 64e943b385514593aa9f495603972282
[  895.509109] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.509310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.509436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.509690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.509838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.510100] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.510258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.510378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.510591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.510837] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.510997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.511115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.511322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.511463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.511611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.511925] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.512114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.512236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.512448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.512586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.512845] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  895.513019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  895.513826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  895.514088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  895.514235] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  895.514466] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1b11680b7e114628a916056477b25607 in queue reply_156db6c4256a4c58884afce15e6f5340
[  895.519395] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1b11680b7e114628a916056477b25607
[  895.655481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  895.656041] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  895.656041] env[60890]: ERROR oslo_messaging.rpc.server 
[  895.858251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  895.858867] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  895.858867] env[60890]: ERROR oslo_messaging.rpc.server 
[  895.986657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  895.987220] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  895.987220] env[60890]: ERROR oslo_messaging.rpc.server 
[  896.016016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  896.016490] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  896.016490] env[60890]: ERROR oslo_messaging.rpc.server 
[  896.190321] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.190535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.190669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.190941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.191094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.191361] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.191523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.191645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.191889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.192161] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.192359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.192498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.192713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.192858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.193020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.193290] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.193452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.193575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.193785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.194461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.194461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.194461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.194461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.194703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.194740] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  896.195017] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a863e76958ca48ae93997cf7c06067e9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  896.199673] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a863e76958ca48ae93997cf7c06067e9
[  896.413530] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.413962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.414256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.416030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.416030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.416330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-801 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.416504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.416629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.416883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.417145] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.417311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.417432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.417656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.417804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.417962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-801" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.418221] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.418378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.418500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.418709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.418846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.419089] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-515e493d-c7c1-4699-a137-1903d3c4f46b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.419244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.419363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.419572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.419707] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('515e493d-c7c1-4699-a137-1903d3c4f46b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  896.419934] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 249d887cf4944af6981972130638d977 in queue reply_156db6c4256a4c58884afce15e6f5340
[  896.425921] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 249d887cf4944af6981972130638d977
[  896.525307] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.525664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.525807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.526102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.526254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.526533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.526697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.526816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.527041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.527325] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.527508] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.527604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.528104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.528104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.528104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.528370] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.528531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.528648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.528889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.528988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.529234] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.529392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.529509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.529742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.529848] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  896.530084] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 88470aae66f84031b73783a458869f79 in queue reply_156db6c4256a4c58884afce15e6f5340
[  896.540527] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 88470aae66f84031b73783a458869f79
[  896.556548] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.556760] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.556894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.557170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.557320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.557577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.557738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.557861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.558086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.558374] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.558537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.558657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.558873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.559019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.559169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.559422] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.559624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.559756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.559969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.560123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.560366] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  896.560525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  896.560670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  896.560890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  896.561029] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  896.561260] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c4ca6692dd28492e932d8ab461a591d9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  896.565846] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c4ca6692dd28492e932d8ab461a591d9
[  896.696843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  896.697363] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  896.697363] env[60890]: ERROR oslo_messaging.rpc.server 
[  896.922162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-515e493d-c7c1-4699-a137-1903d3c4f46b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  896.922660] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  896.922660] env[60890]: ERROR oslo_messaging.rpc.server 
[  897.032227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  897.032749] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  897.032749] env[60890]: ERROR oslo_messaging.rpc.server 
[  897.063546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  897.064090] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  897.064090] env[60890]: ERROR oslo_messaging.rpc.server 
[  897.234711] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.234939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.235074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.235342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.235490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.235747] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.235926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.236073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.236297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.236537] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.236692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.236836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.237014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.237154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.237298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.237544] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.237698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.237815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.238024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.238155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.238391] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.238543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.238660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.238864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.238999] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  897.239226] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9bbe65d7276c42aba9941f18dab3539d in queue reply_156db6c4256a4c58884afce15e6f5340
[  897.243944] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9bbe65d7276c42aba9941f18dab3539d
[  897.617301] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.617632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.617632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.617902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.618051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.618321] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-65 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.618476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.618591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.618820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.619077] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.619233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.619353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.619562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.619709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.619862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-65" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.620146] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.620304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.620424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.620635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.620774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.621019] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-1d11adb4-d495-4272-8cea-9af6747bb237 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.621195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.621315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.621526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.621661] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1d11adb4-d495-4272-8cea-9af6747bb237',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  897.621900] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f996ef5ebddb45d4a517a9cef9380303 in queue reply_156db6c4256a4c58884afce15e6f5340
[  897.627431] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f996ef5ebddb45d4a517a9cef9380303
[  897.634177] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.634377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.634505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.634779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.634974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.635247] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.635408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.635522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.635745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.636299] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.636561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.636738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.637027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.637229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.637406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.637669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.637831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.637950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.638165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.638302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.638549] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.638706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.638825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.639040] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.639183] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  897.639420] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9ba47479db85482ca1d835957bd4c000 in queue reply_156db6c4256a4c58884afce15e6f5340
[  897.644770] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9ba47479db85482ca1d835957bd4c000
[  897.741145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  897.741876] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  897.741876] env[60890]: ERROR oslo_messaging.rpc.server 
[  897.786520] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  897.786682] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  897.925983] env[60890]: INFO networking_nsxv3.common.synchronization [req-ed1b67ca-2b9b-4908-9ed5-413f9130d95f req-129e55eb-db50-45b3-8d9e-77f1ff292e9b tempest-AttachInterfacesTestJSON-1029751346 tempest-AttachInterfacesTestJSON-1029751346-project-member] Enqueued Resource ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 with Priority: HIGHEST for action security_group_members
[  897.925983] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 with Priority: HIGHEST for action security_group_members
[  897.925983] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.926276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.926276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.926378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.926378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.926636] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-b9ba6520-46ed-4da6-847f-1a8360a3f912 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.926636] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.926742] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.926857] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.927066] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  897.927175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  897.927218] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  897.927415] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  897.927538] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  897.927580] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('b9ba6520-46ed-4da6-847f-1a8360a3f912',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  897.927776] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c665c7f4c958425fb54a727495e2b9e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  897.936022] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c665c7f4c958425fb54a727495e2b9e0
[  898.123864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-1d11adb4-d495-4272-8cea-9af6747bb237" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  898.124417] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  898.124417] env[60890]: ERROR oslo_messaging.rpc.server 
[  898.141682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  898.142548] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  898.142548] env[60890]: ERROR oslo_messaging.rpc.server 
[  898.284945] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.285166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.285296] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.285574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.285725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.286011] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.286165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.286290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.286716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.286971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.287134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.287257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.287472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.287617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.287768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.288030] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.288279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.288357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.288515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.288648] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.288888] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.289046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.289169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.289367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.289500] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  898.289739] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5b44b05fa31c41fca99593a27994029f in queue reply_156db6c4256a4c58884afce15e6f5340
[  898.296096] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5b44b05fa31c41fca99593a27994029f
[  898.429928] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.430073] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.430148] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.430372] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.430464] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.430699] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.430790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.430790] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.430939] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.431015] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.431098] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: b9ba6520-46ed-4da6-847f-1a8360a3f912 already deleted.
[  898.431177] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-b9ba6520-46ed-4da6-847f-1a8360a3f912" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.455315] env[60890]: INFO networking_nsxv3.common.synchronization [req-42509e97-6b97-4e92-a0c6-02ef3cc40cf2 req-8e514f75-7abe-48f5-9437-3a775890c8d5 tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 with Priority: HIGHEST for action security_group_members
[  898.455315] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 with Priority: HIGHEST for action security_group_members
[  898.455315] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.455315] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-498c7d18-56d5-4041-ba4b-f3cb998fea54 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.455315] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.455315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.455315] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('498c7d18-56d5-4041-ba4b-f3cb998fea54',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  898.455959] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 33dd5c9ff36f4f24ad86f5db6c683c91 in queue reply_156db6c4256a4c58884afce15e6f5340
[  898.471034] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 33dd5c9ff36f4f24ad86f5db6c683c91
[  898.693154] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.693470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.693505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.693756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.693907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.694166] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.694325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.694452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.694670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.694953] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.695113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.695234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.695490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.695587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.695811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.696111] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.696279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.696402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.696615] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.696754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.696997] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.697161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.697275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.697483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.697618] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  898.697859] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 45acbdfd04554c968c4e5f1623e5ca86 in queue reply_156db6c4256a4c58884afce15e6f5340
[  898.705520] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 45acbdfd04554c968c4e5f1623e5ca86
[  898.791716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  898.792329] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  898.792329] env[60890]: ERROR oslo_messaging.rpc.server 
[  898.958514] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.958674] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.958745] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.958968] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.959062] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.959271] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  898.959373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  898.959436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  898.959591] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  898.959672] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  898.959760] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 already deleted.
[  898.959842] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.032276] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.032425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.032560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.032833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.033002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.033265] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.033422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.033542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.033769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.034014] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.034217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.034289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.034493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.034631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.034782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.035636] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.035636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.035636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.035636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.035966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.036340] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.036584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.037251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.037591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.037792] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  899.038080] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c4b075611e3f478c88d500d56a1a0ea0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  899.044309] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c4b075611e3f478c88d500d56a1a0ea0
[  899.172187] env[60890]: INFO networking_nsxv3.common.synchronization [req-6b5a274b-c1b9-4876-b12f-d6e096c8c8ee req-e27abde0-0c14-4603-a959-87c97a671a64 tempest-ServerRescueTestJSON-523553051 tempest-ServerRescueTestJSON-523553051-project-member] Enqueued Resource ID: ab97351c-3e27-4028-9501-5c8c2ed8e69b with Priority: HIGHEST for action security_group_members
[  899.172187] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ab97351c-3e27-4028-9501-5c8c2ed8e69b with Priority: HIGHEST for action security_group_members
[  899.172479] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.172601] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.172667] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.173077] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.173077] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.173170] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ab97351c-3e27-4028-9501-5c8c2ed8e69b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.173264] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ab97351c-3e27-4028-9501-5c8c2ed8e69b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.173325] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ab97351c-3e27-4028-9501-5c8c2ed8e69b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.173489] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ab97351c-3e27-4028-9501-5c8c2ed8e69b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.173633] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.173746] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.173804] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.173943] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.174016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.174102] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ab97351c-3e27-4028-9501-5c8c2ed8e69b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  899.174274] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4143fab9dd8941129848d92e92c09ccc in queue reply_156db6c4256a4c58884afce15e6f5340
[  899.182088] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4143fab9dd8941129848d92e92c09ccc
[  899.201305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  899.201698] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  899.201698] env[60890]: ERROR oslo_messaging.rpc.server 
[  899.342741] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.343166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.343417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.343712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.343871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.344174] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.344341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.344463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.344699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.345180] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.345357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.345485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.345726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.345879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.346060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.346478] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.346655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.346782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.347006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.347223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.347481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.347642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.347764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.348051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.348144] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  899.348383] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b1836e012bea43f985812e16e36cba1a in queue reply_156db6c4256a4c58884afce15e6f5340
[  899.354619] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b1836e012bea43f985812e16e36cba1a
[  899.540269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  899.540989] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  899.540989] env[60890]: ERROR oslo_messaging.rpc.server 
[  899.677628] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.677628] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.677628] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.677628] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.677628] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.677628] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.677935] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.677935] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.677995] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.678098] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.678142] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ab97351c-3e27-4028-9501-5c8c2ed8e69b already deleted.
[  899.678211] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ab97351c-3e27-4028-9501-5c8c2ed8e69b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.736895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.737185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.737238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.737477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.737624] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.737883] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.738043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.738162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.738382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.738629] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.738784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.738904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.739135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.739447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.739627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.739955] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.740147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.740271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.740487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.741278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.741545] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  899.741711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  899.741836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  899.742091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  899.742236] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('b9d0e96e-98ae-43cb-b514-6a7301fdeeba',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  899.742470] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8c0c412dc00d40a3b6b0e12f6f33bfaf in queue reply_156db6c4256a4c58884afce15e6f5340
[  899.747409] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8c0c412dc00d40a3b6b0e12f6f33bfaf
[  899.851298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  899.851298] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  899.851298] env[60890]: ERROR oslo_messaging.rpc.server 
[  900.097688] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.097900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.098029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.098307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.098453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.098715] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.098873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.098994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.099212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.099461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.099617] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.099736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.099942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.100108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.100259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.100506] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.100661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.100778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.100987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.101123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.101359] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.101663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.101805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.102019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.102155] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  900.102387] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a5b905d6dda14dc28ae4c896dc65023f in queue reply_156db6c4256a4c58884afce15e6f5340
[  900.107393] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a5b905d6dda14dc28ae4c896dc65023f
[  900.246246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-b9d0e96e-98ae-43cb-b514-6a7301fdeeba" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  900.246246] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  900.246246] env[60890]: ERROR oslo_messaging.rpc.server 
[  900.604827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  900.605403] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  900.605403] env[60890]: ERROR oslo_messaging.rpc.server 
[  900.763335] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.763633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.763674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.763929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.764101] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.764360] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.764520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.764640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.764931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.765191] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.765348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.765469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.765676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.765844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.765990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.766239] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.766392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.766511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.766715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.766849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  900.767083] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  900.767262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  900.767383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  900.767591] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  900.767728] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  900.767957] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d9aff23dd7ea447db51197cc4b3cfd3c in queue reply_156db6c4256a4c58884afce15e6f5340
[  900.773758] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d9aff23dd7ea447db51197cc4b3cfd3c
[  901.218963] env[60890]: INFO networking_nsxv3.common.synchronization [req-271d85bf-89bc-41bf-a402-4ab455990cb1 req-3ee5442b-5a95-4d1d-a199-e8fd726265fe tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  901.219689] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  901.220191] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.220382] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.220494] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.220791] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.220937] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.221178] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.221338] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.221445] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.221697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.221921] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.222069] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.222175] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.222391] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.222488] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.222583] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  901.222765] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 403075db088d425c92fab0bfb40474b1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  901.231898] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 403075db088d425c92fab0bfb40474b1
[  901.269885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  901.270406] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  901.270406] env[60890]: ERROR oslo_messaging.rpc.server 
[  901.501329] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.501329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.501329] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.501691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.501794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.502099] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.502278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.502649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.502649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.502901] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.503061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.503179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.503392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.509637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.509896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.510193] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.510357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.510480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.510747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.510898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.511069] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.511226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.511345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.511556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.511693] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  901.512013] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0774716367194361b56da8b809768773 in queue reply_156db6c4256a4c58884afce15e6f5340
[  901.517776] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0774716367194361b56da8b809768773
[  901.728412] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.728412] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.728412] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  901.728412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.820765] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.821070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.821109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.821418] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.821510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.821770] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.821939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.822127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.822376] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.822619] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.822778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.822897] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.823110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.823252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.823402] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.823651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.823808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.823927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.825491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.825603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  901.825858] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  901.826092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  901.826217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  901.826428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  901.826559] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  901.826790] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5adf1317642c4e65af6b7b121554a09a in queue reply_156db6c4256a4c58884afce15e6f5340
[  901.832127] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5adf1317642c4e65af6b7b121554a09a
[  902.016871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  902.017662] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  902.017662] env[60890]: ERROR oslo_messaging.rpc.server 
[  902.328606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  902.329112] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  902.329112] env[60890]: ERROR oslo_messaging.rpc.server 
[  902.579976] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.579976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.579976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.579976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.579976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.580917] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.581107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.581236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.581485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.581757] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.581932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.582130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.582248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.582395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.582564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.582899] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.583002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.583130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.583343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.583480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.583730] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.583884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.584030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.584246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.584443] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  902.584697] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ec9160f281be4b429bd805853670976a in queue reply_156db6c4256a4c58884afce15e6f5340
[  902.590299] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ec9160f281be4b429bd805853670976a
[  902.939354] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.939657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.939657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.939924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.940150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.940349] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.940508] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.940628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.940846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.941085] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.941253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.941357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.941563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.941703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.941850] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.942097] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.942305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.942367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.942563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.942844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  902.942980] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  902.943139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  902.943277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  902.943468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  902.943603] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d92eb678-f5f4-49c3-a3f9-6b39f128a7c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  902.943838] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 190e050cb6e94bf99f14d8d2d9a02483 in queue reply_156db6c4256a4c58884afce15e6f5340
[  902.950252] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 190e050cb6e94bf99f14d8d2d9a02483
[  903.086962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  903.087533] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  903.087533] env[60890]: ERROR oslo_messaging.rpc.server 
[  903.448974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-d92eb678-f5f4-49c3-a3f9-6b39f128a7c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  903.448974] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  903.448974] env[60890]: ERROR oslo_messaging.rpc.server 
[  903.720611] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  903.720828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  903.720962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  903.721255] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  903.721410] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  903.721669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  903.721833] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  903.721958] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  903.722186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  903.722461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  903.722623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  903.722744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  903.722991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  903.723137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  903.723295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  903.723549] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  903.723707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  903.723828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  903.724062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  903.724204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  903.724447] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  903.724606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  903.724724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  903.724965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  903.725101] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  903.725352] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5cf42c37669045ce943ccef3ee5d402e in queue reply_156db6c4256a4c58884afce15e6f5340
[  903.730794] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5cf42c37669045ce943ccef3ee5d402e
[  904.228156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  904.228710] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  904.228710] env[60890]: ERROR oslo_messaging.rpc.server 
[  904.675679] env[60890]: INFO networking_nsxv3.common.synchronization [req-4151299f-90bb-4296-be82-f43d0585b9c9 req-215a1cb8-2612-46bb-b352-e4e2a9c4cd2c tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  904.676171] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  904.676500] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  904.676632] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  904.676701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  904.676894] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  904.676976] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  904.677134] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  904.677226] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  904.677288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  904.677451] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  904.677582] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  904.677678] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  904.677769] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  904.677873] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  904.677953] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  904.678038] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  904.678221] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8465ba5841164cd2b7b44fdf23d80b92 in queue reply_156db6c4256a4c58884afce15e6f5340
[  904.725429] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8465ba5841164cd2b7b44fdf23d80b92
[  904.778155] env[60890]: INFO networking_nsxv3.common.synchronization [None req-032af032-5f78-42f7-ad27-4286c99290f0 None None] Enqueued Resource ID: 607ef90b-1f02-460e-a6f7-4eb8847e650b with Priority: HIGHEST for action port
[  904.778412] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 607ef90b-1f02-460e-a6f7-4eb8847e650b with Priority: HIGHEST for action port
[  904.778966] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  904.779086] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  904.779154] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  904.779482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  904.779482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  904.779585] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-607ef90b-1f02-460e-a6f7-4eb8847e650b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  904.779656] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-607ef90b-1f02-460e-a6f7-4eb8847e650b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  904.779736] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-607ef90b-1f02-460e-a6f7-4eb8847e650b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  904.779939] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-607ef90b-1f02-460e-a6f7-4eb8847e650b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  904.780033] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('607ef90b-1f02-460e-a6f7-4eb8847e650b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  904.780218] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5c19f5842f4e4c789114f86c8d1f1eba in queue reply_156db6c4256a4c58884afce15e6f5340
[  904.791018] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5c19f5842f4e4c789114f86c8d1f1eba
[  905.111545] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.112323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.112323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.112323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.112323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.112628] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.112758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.112875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.113069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.113318] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.113477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.113597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.113808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.113952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.114103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.114359] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.114520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.114641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.115499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.115499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.115499] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.115499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.115690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.115804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.115943] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  905.116196] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cfd22525389b4ab69bed543411042145 in queue reply_156db6c4256a4c58884afce15e6f5340
[  905.121475] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cfd22525389b4ab69bed543411042145
[  905.180489] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.180701] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.180751] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.180930] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.181021] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.181218] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.181315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.181374] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.181521] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.181594] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.181676] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  905.181751] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.285573] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-607ef90b-1f02-460e-a6f7-4eb8847e650b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.285846] env[60890]: Traceback (most recent call last):
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  905.285846] env[60890]:     timer()
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  905.285846] env[60890]:     cb(*args, **kw)
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  905.285846] env[60890]:     result = function(*args, **kwargs)
[  905.285846] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  905.285846] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  905.285846] env[60890]:     return method(*args, **kwargs)
[  905.285846] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  905.285846] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  905.285846] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  905.285846] env[60890]:     result = self.transport._send(
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  905.285846] env[60890]:     return self._driver.send(target, ctxt, message,
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  905.285846] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  905.285846] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  905.285846] env[60890]:     raise result
[  905.285846] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  905.285846] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  905.619139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  905.620296] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  905.620296] env[60890]: ERROR oslo_messaging.rpc.server 
[  905.858774] env[60890]: INFO networking_nsxv3.common.synchronization [None req-8919b65f-d26b-43c8-93db-75e63554046f tempest-ServersTestJSON-605125590 tempest-ServersTestJSON-605125590-project-member] Enqueued Resource ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 with Priority: HIGHEST for action security_group_rules
[  905.859269] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 with Priority: HIGHEST for action security_group_rules
[  905.859586] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.859719] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.859827] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.859979] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.860105] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.860241] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-7dd6a10a-4be3-47d8-a50b-55144390bad6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.860373] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.860436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.860595] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.860677] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('7dd6a10a-4be3-47d8-a50b-55144390bad6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  905.860845] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 23d70c9c409e48bcbdc4c0dabcbc579c in queue reply_156db6c4256a4c58884afce15e6f5340
[  905.875541] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 23d70c9c409e48bcbdc4c0dabcbc579c
[  905.954728] env[60890]: INFO networking_nsxv3.common.synchronization [None req-c59a120c-c470-4377-82a7-c29fd5ff0a13 tempest-ServersTestJSON-605125590 tempest-ServersTestJSON-605125590-project-member] Enqueued Resource ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 with Priority: HIGHEST for action security_group_rules
[  905.955278] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 with Priority: HIGHEST for action security_group_rules
[  905.955981] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.956142] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  905.956434] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  905.956473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  905.956543] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  905.956701] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-7dd6a10a-4be3-47d8-a50b-55144390bad6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  905.956777] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.165849] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.166059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.166185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.166447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.166593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.166917] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.166997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.167159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.167351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.167595] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.167748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.167865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.168094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.168234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.168381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.168630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.168781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.168896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.169104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.169235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.169469] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.169620] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.169734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.170243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.170428] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  906.170667] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2c9825c0866d49f2abacf64d6303c230 in queue reply_156db6c4256a4c58884afce15e6f5340
[  906.176570] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2c9825c0866d49f2abacf64d6303c230
[  906.380484] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.380813] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.380813] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.380889] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.380978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.381074] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 already deleted.
[  906.381149] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.381288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.381482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.381574] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('7dd6a10a-4be3-47d8-a50b-55144390bad6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  906.381769] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 93490198ad6c4530b716518200c3f283 in queue reply_156db6c4256a4c58884afce15e6f5340
[  906.400512] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 93490198ad6c4530b716518200c3f283
[  906.673056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  906.673598] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  906.673598] env[60890]: ERROR oslo_messaging.rpc.server 
[  906.736636] env[60890]: INFO networking_nsxv3.common.synchronization [None req-01b88189-495b-460d-b7f0-2faae30e6fdf None None] Enqueued Resource ID: ff70762e-7fbd-4ffc-b269-5bc9351e371b with Priority: HIGHEST for action port
[  906.736900] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ff70762e-7fbd-4ffc-b269-5bc9351e371b with Priority: HIGHEST for action port
[  906.737507] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.737633] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.737700] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.737899] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.737987] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.738132] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-ff70762e-7fbd-4ffc-b269-5bc9351e371b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.738225] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-ff70762e-7fbd-4ffc-b269-5bc9351e371b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.738284] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-ff70762e-7fbd-4ffc-b269-5bc9351e371b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.738642] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-ff70762e-7fbd-4ffc-b269-5bc9351e371b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.738642] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ff70762e-7fbd-4ffc-b269-5bc9351e371b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  906.739116] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 22bc76e4d2d340f0ad33e93c47263c37 in queue reply_156db6c4256a4c58884afce15e6f5340
[  906.765616] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 22bc76e4d2d340f0ad33e93c47263c37
[  906.901451] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  906.901601] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  906.901671] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  906.901903] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  906.901993] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  906.902085] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 already deleted.
[  906.902163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  907.216291] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  907.216502] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  907.216631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  907.216906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  907.217057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  907.217326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  907.217480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  907.217602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  907.217824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  907.218078] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  907.218234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  907.218354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  907.218563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  907.218705] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  907.218859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  907.219117] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  907.219273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  907.219393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  907.219600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  907.219738] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  907.219978] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  907.220152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  907.220274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  907.220481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  907.220620] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  907.220853] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg aa42f326485e46fb98d6e241f7a0d1bc in queue reply_156db6c4256a4c58884afce15e6f5340
[  907.234557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg aa42f326485e46fb98d6e241f7a0d1bc
[  907.241678] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-ff70762e-7fbd-4ffc-b269-5bc9351e371b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  907.241678] env[60890]: Traceback (most recent call last):
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  907.241678] env[60890]:     timer()
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  907.241678] env[60890]:     cb(*args, **kw)
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  907.241678] env[60890]:     result = function(*args, **kwargs)
[  907.241678] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  907.241678] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  907.241678] env[60890]:     return method(*args, **kwargs)
[  907.241678] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  907.241678] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  907.241678] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  907.241678] env[60890]:     result = self.transport._send(
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  907.241678] env[60890]:     return self._driver.send(target, ctxt, message,
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  907.241678] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  907.241678] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  907.241678] env[60890]:     raise result
[  907.241678] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  907.241678] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  907.729806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  907.730472] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  907.730472] env[60890]: ERROR oslo_messaging.rpc.server 
[  908.277428] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  908.277633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  908.277775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  908.278047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  908.278199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  908.278461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  908.278621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  908.278744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  908.278964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  908.279211] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  908.279366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  908.279485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  908.279691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  908.279832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  908.279984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  908.280262] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  908.280420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  908.280542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  908.280754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  908.280892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  908.281134] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  908.281291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  908.281518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  908.281749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  908.281888] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  908.282118] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d310231c3aaa4d439cef4487a079cb5f in queue reply_156db6c4256a4c58884afce15e6f5340
[  908.287956] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d310231c3aaa4d439cef4487a079cb5f
[  908.783907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  908.784423] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  908.784423] env[60890]: ERROR oslo_messaging.rpc.server 
[  909.630674] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  909.630674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  909.630828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  909.631069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  909.631717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  909.631717] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  909.631717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  909.631717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  909.632068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  909.632246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  909.632403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  909.632520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  909.632747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  909.632898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  909.633048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  909.633301] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  909.633452] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  909.633567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  909.633773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  909.633917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  909.634155] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  909.634306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  909.634421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  909.634622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  909.634753] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ceb8c71b-195d-4a83-ae07-1bdbbcd6f304',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  909.635017] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b3fa407160c142318d7a2cc8a0695f9b in queue reply_156db6c4256a4c58884afce15e6f5340
[  909.644921] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b3fa407160c142318d7a2cc8a0695f9b
[  910.137565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ceb8c71b-195d-4a83-ae07-1bdbbcd6f304" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  910.138081] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  910.138081] env[60890]: ERROR oslo_messaging.rpc.server 
[  911.115381] env[60890]: INFO networking_nsxv3.common.synchronization [req-42509e97-6b97-4e92-a0c6-02ef3cc40cf2 req-53bd1879-cd5b-4609-869e-1b416c3cd0f4 tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 with Priority: HIGHEST for action security_group_members
[  911.116034] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 with Priority: HIGHEST for action security_group_members
[  911.116401] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.116533] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.116604] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.116807] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.116895] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.117052] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-498c7d18-56d5-4041-ba4b-f3cb998fea54 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.117155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.117218] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.117380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.117523] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.117617] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.117678] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.117816] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.117929] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.118025] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('498c7d18-56d5-4041-ba4b-f3cb998fea54',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  911.118209] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 790a5f9903b54d91871c91fbac73a82e in queue reply_156db6c4256a4c58884afce15e6f5340
[  911.126193] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 790a5f9903b54d91871c91fbac73a82e
[  911.588179] env[60890]: INFO networking_nsxv3.common.synchronization [req-e2d19ac7-02f4-468b-97ce-53c340ddbf1b req-27ac3dc4-5f05-49cc-bd5c-ff9cb9c3d38c tempest-ServersTestJSON-605125590 tempest-ServersTestJSON-605125590-project-member] Enqueued Resource ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 with Priority: HIGHEST for action security_group_members
[  911.588687] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 with Priority: HIGHEST for action security_group_members
[  911.589009] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.589204] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.589244] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.589541] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.589584] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.589764] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7dd6a10a-4be3-47d8-a50b-55144390bad6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.589925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.590013] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.590264] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.595320] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.595459] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.595547] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.595803] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.595925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.596019] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7dd6a10a-4be3-47d8-a50b-55144390bad6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  911.596218] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 33e396131b1042a9bd8a8b3adbd2c3fa in queue reply_156db6c4256a4c58884afce15e6f5340
[  911.617873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 33e396131b1042a9bd8a8b3adbd2c3fa
[  911.620145] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.620284] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.620355] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.620556] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.621844] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.622058] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.622166] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.622232] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.622398] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.622479] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.622564] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 498c7d18-56d5-4041-ba4b-f3cb998fea54 already deleted.
[  911.622658] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-498c7d18-56d5-4041-ba4b-f3cb998fea54" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.881596] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.881799] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.882490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.882490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.882490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.882588] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.882734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.882846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.883125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.883592] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.887572] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  911.887572] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  911.887572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  911.888247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  911.889379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  911.889379] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  911.889379] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg adfc080f9d5f43918c47eb06d179ba59 in queue reply_156db6c4256a4c58884afce15e6f5340
[  911.903295] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg adfc080f9d5f43918c47eb06d179ba59
[  912.099887] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  912.099887] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  912.099887] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 already deleted.
[  912.099887] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  912.395029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  912.395634] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  912.395634] env[60890]: ERROR oslo_messaging.rpc.server 
[  912.947492] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  912.947492] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  912.947492] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  912.947492] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  912.947492] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  912.947492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  912.947492] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  912.947492] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0956a8e541af417d9f3dc326082118b0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  912.956257] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0956a8e541af417d9f3dc326082118b0
[  913.448416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  913.448937] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  913.448937] env[60890]: ERROR oslo_messaging.rpc.server 
[  913.988620] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  913.988910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  913.988965] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  913.989217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  913.989366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  913.989629] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  913.989792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  913.989913] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  913.990134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  913.990388] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  913.990552] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  913.990677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  913.990892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  913.991041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  913.991192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  913.991444] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  913.991603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  913.992017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  913.992280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  913.992422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  913.992669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  913.992874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  913.992959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  913.993171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  913.993310] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  913.993544] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9eb3717a2cde448ca3c46c0c7082b304 in queue reply_156db6c4256a4c58884afce15e6f5340
[  913.998455] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9eb3717a2cde448ca3c46c0c7082b304
[  914.496151] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  914.496707] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  914.496707] env[60890]: ERROR oslo_messaging.rpc.server 
[  915.034751] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.035070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.035153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.035421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.035576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.035901] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.036092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.036217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.036440] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.036695] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.036855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.036978] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.037189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.037337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.037485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.037741] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.037899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.038020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.038233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.038369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.038617] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.038770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.038956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.039175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.039314] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  915.039540] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e739061f7e2a4272becb6984dfeaf683 in queue reply_156db6c4256a4c58884afce15e6f5340
[  915.045156] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e739061f7e2a4272becb6984dfeaf683
[  915.143324] env[60890]: INFO networking_nsxv3.common.synchronization [req-89c596fe-9f7e-4b3d-99eb-ff5783b8b68f req-a56d92ad-696c-4cba-8690-c8bb655217a0 tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  915.143814] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  915.144197] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.144357] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.144429] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.144624] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.144709] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.144918] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.145038] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.145088] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.145216] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.145404] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.145453] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.145489] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.145664] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.145705] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.145828] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  915.145955] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9877b9b32a0b4546b83afa43f16b093a in queue reply_156db6c4256a4c58884afce15e6f5340
[  915.160631] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9877b9b32a0b4546b83afa43f16b093a
[  915.417357] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.417567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.417787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.419010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.419010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.419010] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.419179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.419179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.419487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.419815] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.420051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.420179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.420565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.420711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.420864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.421246] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.421407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.421853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.421853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.421853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.422107] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.422499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.422569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.422848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.423001] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  915.423232] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6ad1031819534139984ee2fadda1c879 in queue reply_156db6c4256a4c58884afce15e6f5340
[  915.428774] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6ad1031819534139984ee2fadda1c879
[  915.541666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  915.542194] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  915.542194] env[60890]: ERROR oslo_messaging.rpc.server 
[  915.648155] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.648302] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.648359] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.648561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.648654] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.648854] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  915.648962] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  915.649039] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  915.649194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  915.649274] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.649359] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  915.649438] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.925091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  915.925599] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  915.925599] env[60890]: ERROR oslo_messaging.rpc.server 
[  916.079093] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.079363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.079435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.079704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.079852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.080131] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.080292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.080412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.080629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.080875] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.081033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.081158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.081378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.081521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.081671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.081921] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.082076] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.082194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.082399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.082534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.082772] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.082926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.083041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.083244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.083380] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  916.083781] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ac5f4feaeee8463eb32d0132a5b43599 in queue reply_156db6c4256a4c58884afce15e6f5340
[  916.088778] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ac5f4feaeee8463eb32d0132a5b43599
[  916.464477] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.464877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.465215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.465593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.465884] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.466269] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.466561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.466787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.467135] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.467499] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.467794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.468033] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.468358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.468607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.468860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.469239] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.469528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.469753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.470062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.470310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.470656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  916.470934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  916.471173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  916.471479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  916.471721] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  916.472087] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 59b5a7dc4c8047e9a0f6e20dc9f2fa83 in queue reply_156db6c4256a4c58884afce15e6f5340
[  916.476932] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 59b5a7dc4c8047e9a0f6e20dc9f2fa83
[  916.586240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  916.587508] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  916.587508] env[60890]: ERROR oslo_messaging.rpc.server 
[  916.974377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  916.974946] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  916.974946] env[60890]: ERROR oslo_messaging.rpc.server 
[  917.187687] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.187997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.188060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.188345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.188500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.188782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.188945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.189065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.189291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.189546] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.189708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.189831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.190045] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.190266] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.190443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.190702] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.190861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.190981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.191195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.191334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.191579] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.191737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.191858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.192081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.192221] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  917.192451] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a2f75aae5cde49f797bdba9048f03fa0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  917.198327] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2f75aae5cde49f797bdba9048f03fa0
[  917.363383] env[60890]: INFO networking_nsxv3.common.synchronization [req-e3e0c0df-0222-414c-a6e6-e32b1839b86a req-e7d3d3bb-696c-48dd-a82e-f384132ceb8f tempest-ListServerFiltersTestJSON-931142833 tempest-ListServerFiltersTestJSON-931142833-project-member] Enqueued Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  917.363906] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  917.364243] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.364364] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.364457] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.364653] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.364732] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.364977] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7bb6b0cc-e962-4148-8324-7340c921b84c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.365079] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.365138] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.365349] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.365471] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.365558] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.365614] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.365749] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.365840] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.365925] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7bb6b0cc-e962-4148-8324-7340c921b84c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  917.366668] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 18d7b9230bdd473492f0bdf68da1e43f in queue reply_156db6c4256a4c58884afce15e6f5340
[  917.379811] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 18d7b9230bdd473492f0bdf68da1e43f
[  917.507801] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.508031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.508199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.508477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.508628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.508899] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.509063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.509184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.509406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.509653] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.509810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.509930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.510141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.510283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.510429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.510677] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.510832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.510952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.511160] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.511294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.511531] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.511685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.511805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.512021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.512160] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  917.512389] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2d925296c71342a68723ef27c97e1d3e in queue reply_156db6c4256a4c58884afce15e6f5340
[  917.613287] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2d925296c71342a68723ef27c97e1d3e
[  917.643406] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.643576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.645345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.645345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.645345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.645345] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.645345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.645345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.645345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.645345] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.645345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.645770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.645770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.645770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.645929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.646187] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.646342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.646461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.646670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.646808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.647052] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.647206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.647323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.647702] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.647961] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  917.648230] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 97c13ab823544cc0a34afcb5416a44bc in queue reply_156db6c4256a4c58884afce15e6f5340
[  917.653934] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 97c13ab823544cc0a34afcb5416a44bc
[  917.695408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  917.695970] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  917.695970] env[60890]: ERROR oslo_messaging.rpc.server 
[  917.868749] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.868964] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.869095] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.869400] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.869557] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.869804] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  917.870102] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  917.870221] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  917.870426] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  917.870559] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  917.870808] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7bb6b0cc-e962-4148-8324-7340c921b84c already deleted.
[  917.870808] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.015049] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  918.015641] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  918.015641] env[60890]: ERROR oslo_messaging.rpc.server 
[  918.150386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  918.151000] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  918.151000] env[60890]: ERROR oslo_messaging.rpc.server 
[  918.231760] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.232092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.232092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.232365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.232519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.232824] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.232966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.233068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.233295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.233547] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.233704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.233824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.234036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.234181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.234334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.234651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.234842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.234959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.235177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.235318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.235559] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.235717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.235839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.236088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.236238] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  918.236468] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 02817a68e1d442c8b5b2ee0dcc134821 in queue reply_156db6c4256a4c58884afce15e6f5340
[  918.241905] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 02817a68e1d442c8b5b2ee0dcc134821
[  918.559184] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.559403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.559536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.559818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.559974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.560253] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.560416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.560563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.560916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.561174] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.561332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.561454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.561669] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.561808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.561964] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.562216] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.562374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.562496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.562733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.562874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.563118] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.563276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.563397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.563632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.563769] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  918.563999] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4a80b6b5b7fe4e779329517807ff5cea in queue reply_156db6c4256a4c58884afce15e6f5340
[  918.569306] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4a80b6b5b7fe4e779329517807ff5cea
[  918.686524] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.686728] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.686860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.687132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.687279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.687540] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.687698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.687816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.688051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.688303] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.688463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.688583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.688794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.688935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.689083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.689332] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.689488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.689605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.689812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.689946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.690186] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  918.690339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  918.690456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  918.690663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  918.690797] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  918.691026] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9b937371488349f29efe7671c0d0a112 in queue reply_156db6c4256a4c58884afce15e6f5340
[  918.695845] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9b937371488349f29efe7671c0d0a112
[  918.738811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  918.739255] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  918.739255] env[60890]: ERROR oslo_messaging.rpc.server 
[  919.066081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  919.066587] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  919.066587] env[60890]: ERROR oslo_messaging.rpc.server 
[  919.192921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  919.193425] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  919.193425] env[60890]: ERROR oslo_messaging.rpc.server 
[  919.328518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.328518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.328518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.328518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.328518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.328518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.328518] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  919.328518] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f7a5c6fbbbde4506a67949ca2cb4eff8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  919.333967] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f7a5c6fbbbde4506a67949ca2cb4eff8
[  919.550648] env[60890]: INFO networking_nsxv3.common.synchronization [req-c9a7e8e5-4cd7-42f6-a738-1789f915120f req-8dc35eaa-1af4-44cc-be1a-c6012d44de25 tempest-ListServerFiltersTestJSON-931142833 tempest-ListServerFiltersTestJSON-931142833-project-member] Enqueued Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  919.550648] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  919.550648] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.550648] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7bb6b0cc-e962-4148-8324-7340c921b84c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.550648] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.550648] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.550648] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7bb6b0cc-e962-4148-8324-7340c921b84c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  919.550648] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9d50aed604ff46b399c6bacf219efeea in queue reply_156db6c4256a4c58884afce15e6f5340
[  919.561609] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9d50aed604ff46b399c6bacf219efeea
[  919.613971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.614176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.614303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.614568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.614713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.615036] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.615188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.615310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.615524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.615767] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.615997] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.616149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.616359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.616501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.616646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.616897] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.617052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.617169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.617374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.617509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.617747] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.617900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.618016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.618380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.618380] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  919.618573] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9bc768b4d31045139274711a0bb28364 in queue reply_156db6c4256a4c58884afce15e6f5340
[  919.623373] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9bc768b4d31045139274711a0bb28364
[  919.829433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  919.829957] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  919.829957] env[60890]: ERROR oslo_messaging.rpc.server 
[  919.839494] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.839698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.839826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.845074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.845287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.845575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.845745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.845870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.846123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.846382] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.846542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.846661] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.846876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.847020] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.847173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.847429] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.847633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.847700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.847914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.848128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.848318] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.848478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.848599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.848810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.848951] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  919.849181] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d0d1b0097f7840ce97099d9e09bdeff5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  919.851388] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.851566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.851686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.851908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.852075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.852330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.852488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.854044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.854044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.854044] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.854044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.854044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.854044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.854044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.854044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.854044] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.854594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.854594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.854594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.854746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.854922] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  919.855088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  919.855210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  919.855476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  919.855616] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  919.855836] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7439cfdf9c5d40498313e158773359d2 in queue reply_156db6c4256a4c58884afce15e6f5340
[  919.856743] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d0d1b0097f7840ce97099d9e09bdeff5
[  919.857293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  919.857713] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  919.857713] env[60890]: ERROR oslo_messaging.rpc.server 
[  919.860601] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7439cfdf9c5d40498313e158773359d2
[  920.056128] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.056128] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.056128] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7bb6b0cc-e962-4148-8324-7340c921b84c already deleted.
[  920.056128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.123388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  920.123388] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  920.123388] env[60890]: ERROR oslo_messaging.rpc.server 
[  920.361094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  920.361094] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  920.361094] env[60890]: ERROR oslo_messaging.rpc.server 
[  920.397021] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.397257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.397391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.397664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.397813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.398111] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.398282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.398404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.398676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.398869] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.399030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.399153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.399366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.399510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.399663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.399916] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.400091] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.400215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.400428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.400567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.400824] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.400996] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.401121] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.401335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.401473] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  920.401703] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9e97adcc190f40fd905801cd26c34b04 in queue reply_156db6c4256a4c58884afce15e6f5340
[  920.404293] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.404425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.404549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.404772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.404989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.406521] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.406521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.406521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.406521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.406521] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.406521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.406521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.406521] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.406936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.406936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.407028] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.407130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.407249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.407454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.407586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.407822] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.407991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.408123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.408331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.408461] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  920.408672] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f2eaa9fc38e14d369361705a48cbdf01 in queue reply_156db6c4256a4c58884afce15e6f5340
[  920.409530] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9e97adcc190f40fd905801cd26c34b04
[  920.412934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  920.412934] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  920.412934] env[60890]: ERROR oslo_messaging.rpc.server 
[  920.413878] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f2eaa9fc38e14d369361705a48cbdf01
[  920.413878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  920.414207] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  920.414207] env[60890]: ERROR oslo_messaging.rpc.server 
[  920.659403] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.659618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.659779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.660066] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.660225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.660487] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.660647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.660766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.660982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.661226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.661377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.661494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.661803] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.662059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.662228] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.662540] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.662748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.663053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.663294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.663441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.663703] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.663914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.664052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.664268] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.664402] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  920.664629] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1349a24816a546dbb298f73979f32a35 in queue reply_156db6c4256a4c58884afce15e6f5340
[  920.670365] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1349a24816a546dbb298f73979f32a35
[  920.918265] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.918387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.918433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.918696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.918845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.919101] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.919254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.919366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.919583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.919834] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.919989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.920127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.920335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.920474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.920619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.920871] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.921022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.921139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.921345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.921478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.921718] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.921870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.921987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.922207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.922318] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  920.922546] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 16f9921f9bad4768b9918137b6802c6d in queue reply_156db6c4256a4c58884afce15e6f5340
[  920.928433] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 16f9921f9bad4768b9918137b6802c6d
[  920.977408] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.977719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.977906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.978226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.978436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.978751] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-477 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.978985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.979161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.979429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.979732] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.979962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.980159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.980423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.980623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.980827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-477" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.981130] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.982287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.982439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.982670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.982826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.983085] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.983247] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.983386] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.983575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.983712] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('43d5abcf-dd02-4eb7-92d4-944c2438d6fe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  920.983951] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1b64f3cbb9254ad39922affdd747eae6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  920.985238] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.985448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.985581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.985801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.985949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.986192] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.986350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.986469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.986698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.986951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.987111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.987229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.987436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.987999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.988215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.988586] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.988755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.988877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.989094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.989259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  920.989499] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  920.989654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  920.989772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  920.989982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  920.990115] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  920.990335] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f9f862ffb39247acaed545b8743f1533 in queue reply_156db6c4256a4c58884afce15e6f5340
[  920.992323] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1b64f3cbb9254ad39922affdd747eae6
[  920.995386] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f9f862ffb39247acaed545b8743f1533
[  921.167047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  921.167564] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  921.167564] env[60890]: ERROR oslo_messaging.rpc.server 
[  921.424994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  921.425496] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  921.425496] env[60890]: ERROR oslo_messaging.rpc.server 
[  921.492589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  921.493154] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  921.493154] env[60890]: ERROR oslo_messaging.rpc.server 
[  921.494791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-43d5abcf-dd02-4eb7-92d4-944c2438d6fe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  921.494791] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  921.494791] env[60890]: ERROR oslo_messaging.rpc.server 
[  921.978310] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  921.978523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  921.978655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  921.979027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  921.979180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.979443] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  921.979606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  921.979724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  921.979973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  921.980314] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  921.980675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  921.980675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  921.980761] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  921.980874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.981023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.981321] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  921.981447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  921.981538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  921.981748] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  921.981885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  921.982125] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  921.982279] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  921.982397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  921.982604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  921.982799] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  921.983043] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 18a35235927741d592da831d525412e0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  921.990846] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 18a35235927741d592da831d525412e0
[  922.017741] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.017940] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.018072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.018337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.018483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.018756] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.018935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.019058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.019277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.019526] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.019686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.019807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.020037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.020210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.020378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.020633] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.020793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.020911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.021122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.021258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.021512] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.021691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.021816] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.022029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.022165] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  922.022396] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2dd555fac4b4459ea823e96ac9d39155 in queue reply_156db6c4256a4c58884afce15e6f5340
[  922.029153] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2dd555fac4b4459ea823e96ac9d39155
[  922.072505] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.072688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.072811] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.073078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.073238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.073550] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.073708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.073846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.074248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.074505] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.074692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.074846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.075065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.075207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.075356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.075602] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.075752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.075867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.076132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.076271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.076540] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.076692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.076806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.077010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.077136] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  922.077399] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 97032c466ee74744b9c001f423f7c83a in queue reply_156db6c4256a4c58884afce15e6f5340
[  922.081991] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 97032c466ee74744b9c001f423f7c83a
[  922.210258] env[60890]: INFO networking_nsxv3.common.synchronization [req-023a01cf-8f94-4c1d-a73e-5e345693d7ef req-75928f5e-9739-465a-b8b2-0cc9a66cf15f tempest-ListServerFiltersTestJSON-931142833 tempest-ListServerFiltersTestJSON-931142833-project-member] Enqueued Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  922.215155] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  922.215155] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.215155] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7bb6b0cc-e962-4148-8324-7340c921b84c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.215155] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.215155] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.215155] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7bb6b0cc-e962-4148-8324-7340c921b84c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  922.215155] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5f10da06108f47e7b9620bf01b897112 in queue reply_156db6c4256a4c58884afce15e6f5340
[  922.223734] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5f10da06108f47e7b9620bf01b897112
[  922.316936] env[60890]: INFO networking_nsxv3.common.synchronization [req-e2d19ac7-02f4-468b-97ce-53c340ddbf1b req-29ffea4e-2da4-4899-af87-2cde2148e060 tempest-ServersTestJSON-605125590 tempest-ServersTestJSON-605125590-project-member] Enqueued Resource ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 with Priority: HIGHEST for action security_group_members
[  922.317410] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 with Priority: HIGHEST for action security_group_members
[  922.317726] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.318121] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.318232] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.318449] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.318535] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.318703] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7dd6a10a-4be3-47d8-a50b-55144390bad6 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.318776] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.318838] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.318988] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.319124] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.319213] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.319270] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.319404] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.319482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.319567] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7dd6a10a-4be3-47d8-a50b-55144390bad6',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  922.319738] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 135fd784dea044f78ce5c8cce0727b58 in queue reply_156db6c4256a4c58884afce15e6f5340
[  922.327107] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 135fd784dea044f78ce5c8cce0727b58
[  922.480423] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.480696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.480758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.481025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.481182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.481440] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.481597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.481726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.481942] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.482190] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.482346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.482463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.482673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.482824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.483002] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.483256] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.483411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.483530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.483737] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.483872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.484126] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.484283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.484403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.484662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.484942] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  922.485206] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f69f6adb1e3f4b61b0340d20502a669a in queue reply_156db6c4256a4c58884afce15e6f5340
[  922.487621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  922.488081] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  922.488081] env[60890]: ERROR oslo_messaging.rpc.server 
[  922.491074] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f69f6adb1e3f4b61b0340d20502a669a
[  922.524480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  922.524914] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  922.524914] env[60890]: ERROR oslo_messaging.rpc.server 
[  922.579132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  922.579760] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  922.579760] env[60890]: ERROR oslo_messaging.rpc.server 
[  922.714536] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.714685] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.714748] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.715036] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.715138] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.715331] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.715431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.715492] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.715645] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.715723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.715814] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7bb6b0cc-e962-4148-8324-7340c921b84c already deleted.
[  922.715900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.821432] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.822041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.822041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.822041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.822041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.822318] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  922.822318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  922.822318] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  922.822485] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  922.822485] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.822574] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7dd6a10a-4be3-47d8-a50b-55144390bad6 already deleted.
[  922.822633] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7dd6a10a-4be3-47d8-a50b-55144390bad6" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.987650] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  922.988186] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  922.988186] env[60890]: ERROR oslo_messaging.rpc.server 
[  923.022539] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.023071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.023504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.023853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.024141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.024585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.024908] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.025147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.025482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.027016] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.027323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.027566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.027878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.028177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.028522] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.028885] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.029177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.029405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.029713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.029975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.030320] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.030604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.030828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.031133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.031373] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  923.031707] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ba579e5f362b42bc9417702d9cfa66b3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  923.037908] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ba579e5f362b42bc9417702d9cfa66b3
[  923.085969] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.086342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.086585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.086944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.087322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.087700] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.087987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.088238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.088582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.088936] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.089311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.089914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.090226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.090483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.090734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.091161] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.091457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.091696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.092030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.092285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.092625] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.092910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.093165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.093479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.093848] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  923.094335] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a4da0a3978964d58a153f4a078ef3639 in queue reply_156db6c4256a4c58884afce15e6f5340
[  923.099677] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a4da0a3978964d58a153f4a078ef3639
[  923.117483] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.117823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.118063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.118424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.118678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.119029] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.119314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.119538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.119848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.120233] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.120526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.120749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.121053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.121327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.121585] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.121934] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.122221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.122445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.122839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.123083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.123779] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.124199] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.124454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.124768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.125038] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  923.125376] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6fe9ceb6fc8c48faa18ccae27fada230 in queue reply_156db6c4256a4c58884afce15e6f5340
[  923.131584] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6fe9ceb6fc8c48faa18ccae27fada230
[  923.525232] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.525531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.525569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.525831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.526010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.526274] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.526431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.526551] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.526778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.527019] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.527183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.527297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.527506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.527652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.527801] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.528072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.528233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.528352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.528561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.528696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.528934] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  923.529090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  923.529206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  923.529415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  923.529547] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  923.529774] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 546f81a8c8d24b7c8db1903a7e8e0d25 in queue reply_156db6c4256a4c58884afce15e6f5340
[  923.534385] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 546f81a8c8d24b7c8db1903a7e8e0d25
[  923.534860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  923.535306] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  923.535306] env[60890]: ERROR oslo_messaging.rpc.server 
[  923.596745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  923.597595] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  923.597595] env[60890]: ERROR oslo_messaging.rpc.server 
[  923.627689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  923.628168] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  923.628168] env[60890]: ERROR oslo_messaging.rpc.server 
[  924.032118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  924.032726] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  924.032726] env[60890]: ERROR oslo_messaging.rpc.server 
[  924.072518] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.072742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.072891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.073155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.073304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.073566] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.073723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.073841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.074057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.074300] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.074457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.074577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.074794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.074970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.075124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.075373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.075527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.075644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.075868] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.076023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.076270] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.076425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.076541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.076746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.076897] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  924.077126] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 38c6be3b19384b209b5824e61c633b12 in queue reply_156db6c4256a4c58884afce15e6f5340
[  924.081755] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 38c6be3b19384b209b5824e61c633b12
[  924.166373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.166581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.166709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.167010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.167163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.167420] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.167579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.167697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.167911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.168178] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.168335] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.168453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.168657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.168799] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.168944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.169194] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.169346] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.169501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.169663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.169792] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.170054] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.170214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.170332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.170536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.170669] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  924.170901] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0b3f5998abfa438dbb21c42689a6a83c in queue reply_156db6c4256a4c58884afce15e6f5340
[  924.175939] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0b3f5998abfa438dbb21c42689a6a83c
[  924.178967] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.179145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.179272] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.179492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.179631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.179870] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.180041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.180163] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.180368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.180604] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.180759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.180900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.181079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.181216] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.181363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.181681] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.181842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.181963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.182171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.182304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.182538] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.182700] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.182808] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.183036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.183168] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  924.183379] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ac08b6445504431faa8ed4e259020e28 in queue reply_156db6c4256a4c58884afce15e6f5340
[  924.187612] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ac08b6445504431faa8ed4e259020e28
[  924.518631] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 82b12d062abb48079c2fa16f0c800263 in queue reply_156db6c4256a4c58884afce15e6f5340
[  924.531349] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 82b12d062abb48079c2fa16f0c800263
[  924.570361] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.570578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.570710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.571011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.571170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.571442] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.571605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.571725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.571951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.572221] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.572381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.572500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.572712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.572878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.573043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.573298] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.573454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.573573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.573784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.573920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.574160] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  924.574315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  924.574433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  924.574642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  924.574780] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  924.575032] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d9dc2b6936c0462a80fb7e0bc3e030cb in queue reply_156db6c4256a4c58884afce15e6f5340
[  924.579692] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  924.579692] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  924.579692] env[60890]: ERROR oslo_messaging.rpc.server 
[  924.581110] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d9dc2b6936c0462a80fb7e0bc3e030cb
[  924.673038] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  924.673609] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  924.673609] env[60890]: ERROR oslo_messaging.rpc.server 
[  924.685437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  924.685876] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  924.685876] env[60890]: ERROR oslo_messaging.rpc.server 
[  925.077921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  925.078429] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  925.078429] env[60890]: ERROR oslo_messaging.rpc.server 
[  925.118079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.118381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.118523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.118815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.118976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.119257] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.119425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.119546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.119776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.120037] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.120224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.120321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.120538] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.120680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.120830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.121219] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.121407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.121532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.121747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.121886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.122129] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.122285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.122406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.122616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.122776] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  925.123081] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b8c77ddffaea4d799ab3cd8c9a8de8b3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  925.127829] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b8c77ddffaea4d799ab3cd8c9a8de8b3
[  925.212668] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.212892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.213023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.213292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.213444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.213706] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.213864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.213986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.214221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.214474] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.214631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.214751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.214986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.215133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.215284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.215534] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.215689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.215809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.216039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.216181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.216424] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.216579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.216696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.216904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.217041] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  925.217302] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2c08b0f03fd941e98e52bf3995d76e47 in queue reply_156db6c4256a4c58884afce15e6f5340
[  925.222652] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2c08b0f03fd941e98e52bf3995d76e47
[  925.248513] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.248708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.248834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.249090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.249237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.249529] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.249639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.249758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.250010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.250268] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.250429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.250549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.250759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.250903] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.251052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.251299] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.251455] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.251602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.251782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.251917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.252199] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.252359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.252477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.254215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.254215] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ae1ec1c8-13bd-4b56-8822-8b5e440b9386',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  925.254215] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 17970be21e4c4d068f6ceba8d58750fb in queue reply_156db6c4256a4c58884afce15e6f5340
[  925.258259] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 17970be21e4c4d068f6ceba8d58750fb
[  925.542399] env[60890]: INFO networking_nsxv3.common.synchronization [req-f557cd08-d8cb-41af-b26a-973437cc6e4f req-d127f979-bd4a-4058-97ab-de47f3b28e0d tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  925.542940] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  925.543563] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.543693] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.543761] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.543966] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.544061] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.544221] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.544319] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.544378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.544533] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.544670] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.544760] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.544816] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.544958] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.545040] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.545127] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  925.545305] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2e8c64671f274510806f51ccbc14eb57 in queue reply_156db6c4256a4c58884afce15e6f5340
[  925.552928] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2e8c64671f274510806f51ccbc14eb57
[  925.618165] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.618165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.618165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.618165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.618385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.619182] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.619182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.619182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.619182] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.619373] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.619373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.619457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.620025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.620025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.620025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.620217] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.620379] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.620499] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.620708] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.620841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.621142] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.621309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.621426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.621635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.621764] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  925.622027] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 6c1d9600f3c94ec7815458cfb11e3816 in queue reply_156db6c4256a4c58884afce15e6f5340
[  925.624795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  925.626353] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  925.626353] env[60890]: ERROR oslo_messaging.rpc.server 
[  925.627300] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6c1d9600f3c94ec7815458cfb11e3816
[  925.719712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  925.720112] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  925.720112] env[60890]: ERROR oslo_messaging.rpc.server 
[  925.756497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ae1ec1c8-13bd-4b56-8822-8b5e440b9386" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  925.757019] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  925.757019] env[60890]: ERROR oslo_messaging.rpc.server 
[  925.798038] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.798263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.798392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.798666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.798814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.799140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.799312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.799433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.799670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.799919] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.800107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.800234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.800478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.800627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.800775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.801024] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.801180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.801294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.801494] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.801646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  925.801898] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  925.802047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  925.802166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  925.802430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  925.802566] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  925.802823] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1b18126f871443619495d944d29b0125 in queue reply_156db6c4256a4c58884afce15e6f5340
[  925.810778] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1b18126f871443619495d944d29b0125
[  926.051962] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.052164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.052233] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.052438] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.052532] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.052740] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.052852] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.052914] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.053064] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.053144] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.053233] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  926.053311] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.124206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  926.124736] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  926.124736] env[60890]: ERROR oslo_messaging.rpc.server 
[  926.166951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.167189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.167286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.167950] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.168150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.168417] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.168578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.168701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.168926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.169170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.169322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.169439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.169643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.169782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.169925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.170314] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.170314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.170439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.170626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.170758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.170988] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.171139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.171254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.171460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.171592] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  926.171819] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 782ab260a1034138b3b207fcd0c574cd in queue reply_156db6c4256a4c58884afce15e6f5340
[  926.177222] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 782ab260a1034138b3b207fcd0c574cd
[  926.254215] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.254460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.254592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.254909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.255072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.255334] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.255490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.255608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.255946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.256296] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.256477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.256610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.256837] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.256982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.257158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.257389] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.257545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.257663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.257872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.258005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.258240] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-cc7de68a-e67e-40bf-a22b-e1af02823671 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.258391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.258508] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.258710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.258845] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('cc7de68a-e67e-40bf-a22b-e1af02823671',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  926.259071] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8364203f72b84f56a1f9c54b05c968e3 in queue reply_156db6c4256a4c58884afce15e6f5340
[  926.264313] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8364203f72b84f56a1f9c54b05c968e3
[  926.304477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  926.305149] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  926.305149] env[60890]: ERROR oslo_messaging.rpc.server 
[  926.673472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  926.674080] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  926.674080] env[60890]: ERROR oslo_messaging.rpc.server 
[  926.683075] env[60890]: INFO networking_nsxv3.common.synchronization [req-89c596fe-9f7e-4b3d-99eb-ff5783b8b68f req-8e437faf-bb32-4e8b-bfca-3d5d23147815 tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  926.683630] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  926.683997] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.684195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.684307] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.684544] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.684679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.684906] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.685097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.685201] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.685418] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.685611] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.685797] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.685877] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.686067] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.686194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.686323] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  926.686539] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 58b2b6736ac942819f64d298fd75ee94 in queue reply_156db6c4256a4c58884afce15e6f5340
[  926.693779] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.694034] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.694223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.694506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.694711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.695067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.695301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.695479] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.695745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.696085] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.696314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.696505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.696769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.696971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.697198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.697519] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.697742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.697917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.698192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.698388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.698684] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.698906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.699082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.699397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.703239] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  926.703564] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a9ac3128b980412385f081cf3f86974c in queue reply_156db6c4256a4c58884afce15e6f5340
[  926.704679] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 58b2b6736ac942819f64d298fd75ee94
[  926.709625] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a9ac3128b980412385f081cf3f86974c
[  926.760641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-cc7de68a-e67e-40bf-a22b-e1af02823671" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  926.761137] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  926.761137] env[60890]: ERROR oslo_messaging.rpc.server 
[  926.843424] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.843718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.843911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.844254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.844468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.844784] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.845088] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.845265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.845534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.845844] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.846086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.846259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.846530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.846731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.846936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.847239] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.847523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.847710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.847972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.848195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  926.848489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  926.848706] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  926.848962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  926.849227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  926.849417] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  926.849731] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3f4bb94244f54e94bdeaab4a943337d5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  926.854861] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3f4bb94244f54e94bdeaab4a943337d5
[  927.205172] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.205375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.205420] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.205632] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.205728] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.206067] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.206194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.206266] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.206437] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.206523] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.206616] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  927.206698] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.207179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  927.207651] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  927.207651] env[60890]: ERROR oslo_messaging.rpc.server 
[  927.351292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  927.351790] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  927.351790] env[60890]: ERROR oslo_messaging.rpc.server 
[  927.602372] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.602576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.602709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.602989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.603141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.603403] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.603563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.603682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.603900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.604170] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.604331] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.604453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.604664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.604841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.604989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.605245] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.605477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.605537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.605717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.605873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.606129] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.606348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.606404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.606976] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.607082] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  927.607517] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0e168e5f3b36428e99168d3d40de9bb5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  927.617565] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0e168e5f3b36428e99168d3d40de9bb5
[  927.648528] env[60890]: INFO networking_nsxv3.common.synchronization [None req-2c2d8388-c7ea-4f82-b7f8-a30925e5fdda tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 with Priority: HIGHEST for action security_group_rules
[  927.649021] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 with Priority: HIGHEST for action security_group_rules
[  927.649345] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.649482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.649548] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.649744] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.649821] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.649970] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.650061] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.650118] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.650317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.650388] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('6d7bf239-ec7d-4cfc-b530-3c59651d1c98',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  927.650567] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 91c5a8f95a4d448e820d43f2443d7e5c in queue reply_156db6c4256a4c58884afce15e6f5340
[  927.660473] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 91c5a8f95a4d448e820d43f2443d7e5c
[  927.707871] env[60890]: INFO networking_nsxv3.common.synchronization [req-e3e0c0df-0222-414c-a6e6-e32b1839b86a req-b59ff90b-4a1d-4c89-93ce-6e608b96959c tempest-ListServerFiltersTestJSON-931142833 tempest-ListServerFiltersTestJSON-931142833-project-member] Enqueued Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  927.708170] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  927.708684] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.708946] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.708946] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.709076] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.709128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.709280] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7bb6b0cc-e962-4148-8324-7340c921b84c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.709377] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.709439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.709585] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.709722] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.709812] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.709871] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.710007] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.710084] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.710168] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7bb6b0cc-e962-4148-8324-7340c921b84c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  927.710350] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0fd8a8161110446fa0722183fe127327 in queue reply_156db6c4256a4c58884afce15e6f5340
[  927.716443] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.716677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.716733] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.716953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.717092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.717337] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.717490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.717986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.717986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.718143] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.718236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.718290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.719117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.719117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.719117] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.719117] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.719391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.719391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.719500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.719557] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.719782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.719939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.720070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.720273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.720399] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  927.720612] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2f43807132104193a461bc3a500429ee in queue reply_156db6c4256a4c58884afce15e6f5340
[  927.721409] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0fd8a8161110446fa0722183fe127327
[  927.726121] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2f43807132104193a461bc3a500429ee
[  927.782048] env[60890]: INFO networking_nsxv3.common.synchronization [None req-a509d820-3b19-478e-8a00-9cf11ee6b6a0 tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 with Priority: HIGHEST for action security_group_rules
[  927.782370] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 with Priority: HIGHEST for action security_group_rules
[  927.782977] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 230 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.783151] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.783281] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.783532] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.783668] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.783889] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 233 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.784041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.815080] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.815401] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.815560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.816094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.816264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.816533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.816695] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.816817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.817276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.817537] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.817701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.817921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.818032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.818460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.818641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.818902] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.819062] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.819181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.819556] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.819731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  927.819986] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  927.820173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  927.820863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  927.821114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  927.821373] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  927.821651] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b31ccd75ba5d4457b5fc49d4f4eed080 in queue reply_156db6c4256a4c58884afce15e6f5340
[  927.827292] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b31ccd75ba5d4457b5fc49d4f4eed080
[  928.109127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  928.109665] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  928.109665] env[60890]: ERROR oslo_messaging.rpc.server 
[  928.171213] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.171455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.171455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.171642] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.171731] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.171825] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 already deleted.
[  928.171908] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.172063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.172222] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.172304] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_group called with arguments ('6d7bf239-ec7d-4cfc-b530-3c59651d1c98',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  928.172488] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e5606e39a4b24fafb223ccb6964c8986 in queue reply_156db6c4256a4c58884afce15e6f5340
[  928.182061] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e5606e39a4b24fafb223ccb6964c8986
[  928.212350] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.212499] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.212561] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.213242] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.213439] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.213628] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.213737] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.213799] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.214315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.214413] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.214539] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7bb6b0cc-e962-4148-8324-7340c921b84c already deleted.
[  928.214595] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.223848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  928.223848] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  928.223848] env[60890]: ERROR oslo_messaging.rpc.server 
[  928.323921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  928.324542] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  928.324542] env[60890]: ERROR oslo_messaging.rpc.server 
[  928.669842] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.670047] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.670177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.671719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.671719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.671719] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.671719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.671719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.671719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.671719] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.671719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.672056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.672056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.672139] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.672285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.672536] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.672694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.672861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.673081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.673217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.673497] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.673606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.673724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.673932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.674069] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  928.674297] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5320d79920814e2e97df108a41d36555 in queue reply_156db6c4256a4c58884afce15e6f5340
[  928.679512] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5320d79920814e2e97df108a41d36555
[  928.695863] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 257 in security_group_rules>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 886 in sg_rules_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.695863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.695863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.695863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.695863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.695863] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Rules) with ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 already deleted.
[  928.695863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "rules-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.765473] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.765774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.765907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.766277] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.766491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.766860] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.767133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.767262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.767570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.767938] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.768190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.768377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.768679] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.768885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.769111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.769461] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.769684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.769858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.770153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.770352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.770759] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.770925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.771105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.771398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.771593] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  928.771923] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8dc3c79f388841899dc1a728f8116a57 in queue reply_156db6c4256a4c58884afce15e6f5340
[  928.777278] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8dc3c79f388841899dc1a728f8116a57
[  928.863596] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.863596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.863596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.863596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.863596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.863596] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.863596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.863596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.863972] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.863972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.864073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.864193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.864406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.864550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.864699] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.865043] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.865235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.865299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.865511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.865644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  928.865924] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  928.866035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  928.866152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  928.866359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  928.866491] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  928.866723] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8a12948e486e44c8aaae8d3a47f2880d in queue reply_156db6c4256a4c58884afce15e6f5340
[  928.871968] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8a12948e486e44c8aaae8d3a47f2880d
[  929.177392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  929.177916] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  929.177916] env[60890]: ERROR oslo_messaging.rpc.server 
[  929.274794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  929.275347] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  929.275347] env[60890]: ERROR oslo_messaging.rpc.server 
[  929.369371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  929.369885] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  929.369885] env[60890]: ERROR oslo_messaging.rpc.server 
[  929.711015] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.711233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.711363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.711634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.711783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.712093] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.712456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.712456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.712633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.712898] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.713056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.713176] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.713387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.713530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.713676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.714048] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.714130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.714190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.714400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.714537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.714775] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-c13d3a36-2ac2-466f-83c1-7462857ec45e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.714959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.715082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.715348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.715419] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c13d3a36-2ac2-466f-83c1-7462857ec45e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  929.715648] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0a33bd83dedb4f85af84de123114075a in queue reply_156db6c4256a4c58884afce15e6f5340
[  929.721184] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0a33bd83dedb4f85af84de123114075a
[  929.839987] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.840292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.840358] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.840612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.840759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.841072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.841232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.841354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.841573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.841818] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.841975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.842093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.842299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.842438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.842586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.842853] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.843028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.843150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.843359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.843496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.843731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.843887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.844044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.844233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.844370] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  929.844596] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0177b0e9a4c2422ca4aaee223b70365d in queue reply_156db6c4256a4c58884afce15e6f5340
[  929.849557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0177b0e9a4c2422ca4aaee223b70365d
[  929.921140] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.921345] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.921474] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.921745] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.921896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.922156] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.922313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.922433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.922643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.922991] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.923156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.923275] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.923483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.923626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.923777] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.924047] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.924209] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.924330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.924535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.924671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  929.924972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  929.925134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  929.925254] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  929.925462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  929.925597] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  929.925856] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a79aac137d2b4a65bd482d06a3c9fdee in queue reply_156db6c4256a4c58884afce15e6f5340
[  929.931274] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a79aac137d2b4a65bd482d06a3c9fdee
[  930.218332] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-c13d3a36-2ac2-466f-83c1-7462857ec45e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  930.218842] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  930.218842] env[60890]: ERROR oslo_messaging.rpc.server 
[  930.347110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  930.347793] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  930.347793] env[60890]: ERROR oslo_messaging.rpc.server 
[  930.427901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  930.428427] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  930.428427] env[60890]: ERROR oslo_messaging.rpc.server 
[  930.975266] env[60890]: INFO networking_nsxv3.common.synchronization [req-c9a7e8e5-4cd7-42f6-a738-1789f915120f req-144f7936-8395-44c3-97bc-7f28f640e11b tempest-ListServerFiltersTestJSON-931142833 tempest-ListServerFiltersTestJSON-931142833-project-member] Enqueued Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  930.975831] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  930.976146] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  930.976276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  930.976343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  930.976537] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  930.976623] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  930.976769] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7bb6b0cc-e962-4148-8324-7340c921b84c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  930.976867] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  930.976928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  930.977081] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  930.977228] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  930.977315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  930.977371] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  930.977506] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  930.977582] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  930.977669] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7bb6b0cc-e962-4148-8324-7340c921b84c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  930.977847] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 41e902558b2d4aab93cc7a07ca24ee05 in queue reply_156db6c4256a4c58884afce15e6f5340
[  930.985952] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 41e902558b2d4aab93cc7a07ca24ee05
[  931.024493] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.024845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.024845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.025124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.025274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.025535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.025694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.025892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.026099] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.026359] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.026539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.026663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.026876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.027019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.027192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.027439] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.027562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.027682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.027889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.028043] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.028289] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.028444] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.028562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.028767] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.028901] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  931.029132] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 92b029c93a034b2f8c515fec426b47b5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  931.032226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.032399] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.032526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.032749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.032891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.033228] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.033310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.033418] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.033625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.034757] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.034757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.034757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.034757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.034757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.034757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.035032] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.035032] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.035113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.035459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.035459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.035688] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.035853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.035995] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.036304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.036441] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  931.036647] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2861692d79504f5c83c4d21ff2aa6761 in queue reply_156db6c4256a4c58884afce15e6f5340
[  931.037475] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 92b029c93a034b2f8c515fec426b47b5
[  931.040772] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2861692d79504f5c83c4d21ff2aa6761
[  931.479476] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.479655] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.479716] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.479937] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.480045] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.480241] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  931.480341] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  931.480401] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  931.480550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  931.480631] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.480740] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7bb6b0cc-e962-4148-8324-7340c921b84c already deleted.
[  931.480837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.531352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  931.531869] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  931.531869] env[60890]: ERROR oslo_messaging.rpc.server 
[  931.538839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  931.539278] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  931.539278] env[60890]: ERROR oslo_messaging.rpc.server 
[  932.109548] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.109548] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-398 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.109548] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-398" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.109548] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.109548] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.109548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.109548] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4baa1cf4-6b34-483b-bbb4-ea0a83443dac',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  932.109548] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 0061e042f3544b83aca8e6c47e66e330 in queue reply_156db6c4256a4c58884afce15e6f5340
[  932.113493] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0061e042f3544b83aca8e6c47e66e330
[  932.380091] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.380515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.380815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.381215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.381510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.381903] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.382222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.382595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.382952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.383603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.383953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.384251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.384593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.384895] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.385186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.385568] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.385943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.386218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.386553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.386836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.387226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  932.387563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  932.387830] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  932.388183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  932.388458] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  932.388817] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 58e8067c90dc48a8a202392a2aee7919 in queue reply_156db6c4256a4c58884afce15e6f5340
[  932.393893] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 58e8067c90dc48a8a202392a2aee7919
[  932.610593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-4baa1cf4-6b34-483b-bbb4-ea0a83443dac" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  932.611130] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  932.611130] env[60890]: ERROR oslo_messaging.rpc.server 
[  932.891420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  932.891975] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  932.891975] env[60890]: ERROR oslo_messaging.rpc.server 
[  933.431446] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  933.431755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  933.431810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  933.432067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  933.432222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  933.432486] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  933.432645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  933.432782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  933.433011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  933.433847] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  933.433847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  933.433847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  933.433847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  933.433847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  933.434133] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  933.434391] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  933.434462] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  933.434577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  933.434840] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  933.434980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  933.435226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  933.435381] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  933.435501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  933.435709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  933.435849] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  933.436098] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8134a0c7af16480fb12ef86e6cc27314 in queue reply_156db6c4256a4c58884afce15e6f5340
[  933.442518] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8134a0c7af16480fb12ef86e6cc27314
[  933.906311] env[60890]: INFO networking_nsxv3.common.synchronization [req-023a01cf-8f94-4c1d-a73e-5e345693d7ef req-7546e8a9-41f1-4ca3-b4d0-91969dc31ea5 tempest-ListServerFiltersTestJSON-931142833 tempest-ListServerFiltersTestJSON-931142833-project-member] Enqueued Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  933.906871] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7bb6b0cc-e962-4148-8324-7340c921b84c with Priority: HIGHEST for action security_group_members
[  933.907186] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  933.907313] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  933.907379] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  933.907572] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  933.907656] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  933.907814] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-7bb6b0cc-e962-4148-8324-7340c921b84c taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  933.907932] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  933.907991] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  933.908158] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  933.908294] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  933.908387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  933.908455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  933.908576] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  933.908653] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  933.908737] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('7bb6b0cc-e962-4148-8324-7340c921b84c',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  933.908907] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3c2946427ae74cc5862b4515cfd95521 in queue reply_156db6c4256a4c58884afce15e6f5340
[  933.916991] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3c2946427ae74cc5862b4515cfd95521
[  933.938167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  933.938667] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  933.938667] env[60890]: ERROR oslo_messaging.rpc.server 
[  934.412411] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.412411] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.412411] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 7bb6b0cc-e962-4148-8324-7340c921b84c already deleted.
[  934.412411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-7bb6b0cc-e962-4148-8324-7340c921b84c" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.506412] env[60890]: INFO networking_nsxv3.common.synchronization [req-b157fe83-0138-4cbc-a20c-a7355a9a9feb req-d6ffa6ca-7f58-4b96-ae1f-1765a8be505d tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 with Priority: HIGHEST for action security_group_members
[  934.506412] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 with Priority: HIGHEST for action security_group_members
[  934.506412] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.506412] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.506412] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.506412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.506412] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('6d7bf239-ec7d-4cfc-b530-3c59651d1c98',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  934.506412] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d21970eb37df4a4f83a955fc3c89dc46 in queue reply_156db6c4256a4c58884afce15e6f5340
[  934.514897] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d21970eb37df4a4f83a955fc3c89dc46
[  934.545481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.545481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.545481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.545481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.545481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.545481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.545481] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  934.545481] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 88d9bf80dfa74ab5b4226b58438f3d49 in queue reply_156db6c4256a4c58884afce15e6f5340
[  934.554086] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 88d9bf80dfa74ab5b4226b58438f3d49
[  934.780258] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.780258] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.780258] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.780258] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.780258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  934.781072] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  934.781145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  934.782039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  934.782039] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  934.782039] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  934.782039] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 511fca7edeb34adc911b302ffbb6fe25 in queue reply_156db6c4256a4c58884afce15e6f5340
[  934.788917] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 511fca7edeb34adc911b302ffbb6fe25
[  935.008819] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.008992] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.009064] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.009280] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.009372] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.009567] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.009664] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.009724] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.009871] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.009947] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.010030] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 already deleted.
[  935.010108] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.047823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  935.048380] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  935.048380] env[60890]: ERROR oslo_messaging.rpc.server 
[  935.287233] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  935.287753] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  935.287753] env[60890]: ERROR oslo_messaging.rpc.server 
[  935.599749] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.600134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.600169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.600464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.600619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.600894] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.601059] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.601181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.601404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.601653] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.601812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.601933] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.602143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.602292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.602445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.602696] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.602894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.603028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.603242] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.603373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.603674] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-3a2b8891-987c-4ef4-adf1-5c9482a029c9 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.603757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.603871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.604090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.604223] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('3a2b8891-987c-4ef4-adf1-5c9482a029c9',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  935.604459] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 74b0eb9035c24f42996e04287a184359 in queue reply_156db6c4256a4c58884afce15e6f5340
[  935.609713] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 74b0eb9035c24f42996e04287a184359
[  935.847846] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.848074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.848215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.848482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.848663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.848938] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.849092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.849206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.849424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.849663] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.849810] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.849922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.850193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.850295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.850441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.850799] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.850961] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.851073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.851281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.851445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  935.851638] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  935.851785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  935.851907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  935.852175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  935.852340] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  935.852653] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1a9ea2b114694e8191bab981fb946de5 in queue reply_156db6c4256a4c58884afce15e6f5340
[  935.857540] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1a9ea2b114694e8191bab981fb946de5
[  936.106680] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-3a2b8891-987c-4ef4-adf1-5c9482a029c9" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  936.107230] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  936.107230] env[60890]: ERROR oslo_messaging.rpc.server 
[  936.355323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  936.355864] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  936.355864] env[60890]: ERROR oslo_messaging.rpc.server 
[  936.888160] env[60890]: INFO networking_nsxv3.common.synchronization [req-f557cd08-d8cb-41af-b26a-973437cc6e4f req-32844319-04d3-4887-a73c-438093f0b371 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  936.888670] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  936.888981] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  936.889104] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  936.889168] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  936.889355] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  936.889435] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  936.889578] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  936.889669] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  936.889730] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  936.889868] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  936.890007] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  936.890131] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  936.890166] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  936.890283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  936.890360] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  936.890443] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  936.890617] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 210e8b44c1bd44f5a2957febffe555ef in queue reply_156db6c4256a4c58884afce15e6f5340
[  936.909561] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 210e8b44c1bd44f5a2957febffe555ef
[  936.937245] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  936.937451] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  936.937579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  936.937848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  936.938000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  936.938262] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  936.938421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  936.938541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  936.938757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  936.939003] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  936.939159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  936.939278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  936.939531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  936.939622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  936.939785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  936.940073] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  936.940244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  936.940365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  936.940574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  936.940722] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  936.940970] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  936.941129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  936.941246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  936.941458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  936.941589] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  936.941818] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d37023f27a4640f49908777fc195b434 in queue reply_156db6c4256a4c58884afce15e6f5340
[  936.946714] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d37023f27a4640f49908777fc195b434
[  937.392250] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  937.392641] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  937.392641] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  937.392773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  937.392773] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  937.393000] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  937.393101] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  937.393164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  937.393315] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  937.393392] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  937.393475] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  937.393553] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  937.443280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  937.443762] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  937.443762] env[60890]: ERROR oslo_messaging.rpc.server 
[  938.020720] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  938.021017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  938.021060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  938.021320] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  938.021461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  938.021718] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  938.021901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  938.022022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  938.022235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  938.022475] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  938.022626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  938.022766] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  938.023006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  938.023145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  938.023289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  938.023540] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  938.023689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  938.023804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  938.024021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  938.024158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  938.024395] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  938.024545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  938.024660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  938.024912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  938.025059] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  938.025292] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg fa133a8266bf4b569ccd267d3a43ec3e in queue reply_156db6c4256a4c58884afce15e6f5340
[  938.030182] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg fa133a8266bf4b569ccd267d3a43ec3e
[  938.527442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  938.528023] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  938.528023] env[60890]: ERROR oslo_messaging.rpc.server 
[  939.069309] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  939.069593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  939.069638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  939.069910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  939.070064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  939.070323] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  939.070486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  939.070609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  939.070831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  939.071081] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  939.071241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  939.071363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  939.071572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  939.071717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  939.071870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  939.072141] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  939.072301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  939.072421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  939.072630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  939.072769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  939.073010] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  939.073170] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  939.073290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  939.073504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  939.073641] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  939.073873] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7066524ca31c4bd08de38f749e447cf0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  939.080520] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7066524ca31c4bd08de38f749e447cf0
[  939.575535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  939.576307] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  939.576307] env[60890]: ERROR oslo_messaging.rpc.server 
[  940.114450] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  940.114779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  940.114779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  940.115106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  940.115262] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  940.115530] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  940.115691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  940.115817] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  940.116051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  940.116305] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  940.116464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  940.116584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  940.116795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  940.116937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  940.117087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  940.117372] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  940.117529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  940.117649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  940.117861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  940.118000] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  940.118238] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  940.118392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  940.118548] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  940.118715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  940.118849] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  940.119078] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c41f732d1f21489caba11e5dc252f567 in queue reply_156db6c4256a4c58884afce15e6f5340
[  940.123812] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c41f732d1f21489caba11e5dc252f567
[  940.621725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  940.622269] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  940.622269] env[60890]: ERROR oslo_messaging.rpc.server 
[  941.160877] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.161150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.161230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.161498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.161645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.161905] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.162064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.162183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.162400] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.162645] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.162800] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.162917] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.163125] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.163264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.163411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.163661] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.163819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.163939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.164168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.164305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.164540] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.164696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.164852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.165061] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.165199] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  941.165429] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 415f1ae5576e483b82c0634f59da44ab in queue reply_156db6c4256a4c58884afce15e6f5340
[  941.170455] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 415f1ae5576e483b82c0634f59da44ab
[  941.539723] env[60890]: INFO networking_nsxv3.common.synchronization [req-092c8aa4-2449-4c2f-8499-2089a162b25c req-667e90c0-7388-4c22-a296-6e329d944699 tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  941.539723] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  941.539723] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.539723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.539723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.539723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.539723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.539723] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.539723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.539723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.539723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.539723] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.539723] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.540614] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.540614] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.540614] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.540720] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  941.540980] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 81edfe13341b4c8a8254cdd3d7c7b463 in queue reply_156db6c4256a4c58884afce15e6f5340
[  941.553487] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 81edfe13341b4c8a8254cdd3d7c7b463
[  941.667566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  941.668232] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  941.668232] env[60890]: ERROR oslo_messaging.rpc.server 
[  941.811362] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.811568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.811696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.811974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.812144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.813319] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.813319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.813319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.813319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.813319] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.813319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.813646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.813646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.813764] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.813881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.815423] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.815423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.815423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.815423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.815423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  941.815423] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  941.815423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  941.815423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  941.815802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  941.815802] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  941.815982] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ba292ca0eb874f12b2cae928042a87de in queue reply_156db6c4256a4c58884afce15e6f5340
[  941.821453] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ba292ca0eb874f12b2cae928042a87de
[  942.047409] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.047562] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.047562] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.047771] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.047861] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.048063] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.048160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.048223] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.048365] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.048441] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.048523] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  942.048596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.207106] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.207426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.207573] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.207841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.208016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.208291] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.208450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.208571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.208794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.209046] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.209211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.209324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.209534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.209677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.209824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.210077] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.210232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.210350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.210558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.210693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.210934] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.211090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.211207] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.211413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.211549] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  942.211774] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ce57c2db49244ebd80257b951317bdaa in queue reply_156db6c4256a4c58884afce15e6f5340
[  942.216920] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ce57c2db49244ebd80257b951317bdaa
[  942.317827] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  942.318484] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  942.318484] env[60890]: ERROR oslo_messaging.rpc.server 
[  942.714143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  942.714655] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  942.714655] env[60890]: ERROR oslo_messaging.rpc.server 
[  942.855882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.856110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.856244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.856515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.856664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.856922] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.857081] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.857200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.857422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.857667] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.857840] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.857952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.858146] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.858285] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.858478] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.858681] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.858832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.858953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.859158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.859294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  942.859530] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  942.859686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  942.859805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  942.860096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  942.860191] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  942.860424] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 70f830257fb3416d8540c074be4be623 in queue reply_156db6c4256a4c58884afce15e6f5340
[  942.866587] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 70f830257fb3416d8540c074be4be623
[  943.251288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.251580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.251621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.251878] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.252115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.252381] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.252537] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.252655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.252912] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.253160] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.253315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.253432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.253640] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.253781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.253925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.254175] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.254330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.254445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.254649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.254858] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.255086] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.255240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.255357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.255561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.255721] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  943.256084] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 20caa9ce38df4767b65378c52d092140 in queue reply_156db6c4256a4c58884afce15e6f5340
[  943.261761] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 20caa9ce38df4767b65378c52d092140
[  943.363257] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  943.363776] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  943.363776] env[60890]: ERROR oslo_messaging.rpc.server 
[  943.758856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  943.759375] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  943.759375] env[60890]: ERROR oslo_messaging.rpc.server 
[  943.901298] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.901543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.901676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.901953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.902102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.902362] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.902514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.902633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.902846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.903090] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.903245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.903363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.903570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.903709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.903876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.904450] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.904450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.904576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.904676] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.904911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  943.905164] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  943.905323] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  943.905442] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  943.905652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  943.905824] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  943.906018] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d4d4d5b6cea34e4291dfb25ad92ab861 in queue reply_156db6c4256a4c58884afce15e6f5340
[  943.910807] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d4d4d5b6cea34e4291dfb25ad92ab861
[  944.297376] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.297659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.297711] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.297973] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.298126] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.298382] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-851 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.298539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.298660] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.298882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.299130] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.299284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.299403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.299612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.299751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.299899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-851" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.300173] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.300328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.300445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.300655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.300791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.301029] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-53161009-b0b1-432b-874f-97f873168b97 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.301186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.301302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.301511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.301645] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('53161009-b0b1-432b-874f-97f873168b97',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  944.301877] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bfbcdcf782094206bb05fecaab798a93 in queue reply_156db6c4256a4c58884afce15e6f5340
[  944.306720] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bfbcdcf782094206bb05fecaab798a93
[  944.407921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  944.408628] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  944.408628] env[60890]: ERROR oslo_messaging.rpc.server 
[  944.804353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-53161009-b0b1-432b-874f-97f873168b97" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  944.804948] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  944.804948] env[60890]: ERROR oslo_messaging.rpc.server 
[  944.946263] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.946469] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.946598] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.946932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.947096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.947387] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.947553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.947673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.947901] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.948162] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.948318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.948438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.948642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.948783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.948932] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.949180] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.949333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.949450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.949654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.949786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  944.950091] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  944.950248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  944.950366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  944.950574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  944.950708] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  944.950930] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 311b6e6c780b4b82b6a746cca167c397 in queue reply_156db6c4256a4c58884afce15e6f5340
[  944.955666] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 311b6e6c780b4b82b6a746cca167c397
[  945.453092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  945.453610] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  945.453610] env[60890]: ERROR oslo_messaging.rpc.server 
[  945.609024] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  945.609162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  945.609291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  945.614437] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  945.614437] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  945.614437] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  945.614437] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  945.614437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  945.614437] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  945.614437] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 820596e2cb184a0d9c83dee2cc33c924 in queue reply_156db6c4256a4c58884afce15e6f5340
[  945.615689] env[60890]: INFO networking_nsxv3.common.synchronization [req-b157fe83-0138-4cbc-a20c-a7355a9a9feb req-8583cd76-e334-4628-9c8e-1bd26aa7465e tempest-AttachVolumeNegativeTest-1295584799 tempest-AttachVolumeNegativeTest-1295584799-project-member] Enqueued Resource ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 with Priority: HIGHEST for action security_group_members
[  945.615918] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 with Priority: HIGHEST for action security_group_members
[  945.616489] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  945.616489] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  945.616489] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  945.616622] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  945.616697] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  945.616833] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  945.616919] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  945.617104] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  945.618092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  945.618092] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  945.618092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  945.618092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  945.618092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  945.618092] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  945.618092] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('6d7bf239-ec7d-4cfc-b530-3c59651d1c98',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  945.618092] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 9f1096915432412896de403fb777c542 in queue reply_156db6c4256a4c58884afce15e6f5340
[  945.618757] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 820596e2cb184a0d9c83dee2cc33c924
[  945.633828] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9f1096915432412896de403fb777c542
[  946.116467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  946.117074] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  946.117074] env[60890]: ERROR oslo_messaging.rpc.server 
[  946.120051] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  946.120363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  946.120363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  946.120518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  946.120607] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.120816] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  946.121050] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  946.121119] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  946.121431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  946.121539] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.121630] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: 6d7bf239-ec7d-4cfc-b530-3c59651d1c98 already deleted.
[  946.121736] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-6d7bf239-ec7d-4cfc-b530-3c59651d1c98" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.300033] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  946.300271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  946.300405] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  946.300673] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  946.300823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.301092] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  946.301249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  946.301371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  946.301592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  946.301839] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  946.302011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  946.302169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  946.302326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  946.302467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.302618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.302872] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  946.303030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  946.303150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  946.303361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  946.303497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.303887] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  946.304068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  946.304194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  946.304441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  946.304613] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  946.304927] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f91c16d216844813a9d898962ed9cbbd in queue reply_156db6c4256a4c58884afce15e6f5340
[  946.309983] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f91c16d216844813a9d898962ed9cbbd
[  946.806871] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  946.807428] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  946.807428] env[60890]: ERROR oslo_messaging.rpc.server 
[  947.412394] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  947.412637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  947.412849] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  947.413144] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  947.413289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  947.413557] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  947.413713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  947.413834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  947.414053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  947.414374] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  947.414540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  947.414662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  947.414937] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  947.415094] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  947.415248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  947.415555] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  947.415853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  947.415952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  947.416164] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  947.416278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  947.416522] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  947.416677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  947.416795] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  947.417012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  947.417172] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  947.417409] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d2f0685b0cd34bb787acec5b5b2040fc in queue reply_156db6c4256a4c58884afce15e6f5340
[  947.422309] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d2f0685b0cd34bb787acec5b5b2040fc
[  947.919397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  947.919915] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  947.919915] env[60890]: ERROR oslo_messaging.rpc.server 
[  948.456601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  948.456829] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  948.456979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  948.457394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  948.457563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  948.457842] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  948.458012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  948.458137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  948.458374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  948.458624] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  948.458785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  948.458907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  948.459119] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  948.459265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  948.459416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  948.459666] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  948.459824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  948.459946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  948.460177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  948.460319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  948.460556] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  948.460873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  948.460873] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  948.461026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  948.461158] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  948.461455] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b9d61efd89a646dda34f85ce3f46f398 in queue reply_156db6c4256a4c58884afce15e6f5340
[  948.467324] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b9d61efd89a646dda34f85ce3f46f398
[  948.964780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  948.965351] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  948.965351] env[60890]: ERROR oslo_messaging.rpc.server 
[  949.500569] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  949.500782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  949.500914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  949.501211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  949.501338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  949.501590] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  949.501749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  949.501870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  949.502086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  949.502336] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  949.502491] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  949.502611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  949.502819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  949.502959] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  949.503104] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  949.503352] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  949.503512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  949.503630] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  949.503867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  949.504027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  949.504269] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  949.504423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  949.504542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  949.504749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  949.504953] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  949.505219] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg eac273e2f77949eebcef6e86588cfb18 in queue reply_156db6c4256a4c58884afce15e6f5340
[  949.510591] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eac273e2f77949eebcef6e86588cfb18
[  950.007659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  950.008724] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  950.008724] env[60890]: ERROR oslo_messaging.rpc.server 
[  950.546190] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  950.546392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  950.546520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  950.546794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  950.546951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  950.547230] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  950.547375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  950.547489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  950.547867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  950.548171] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  950.548348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  950.548475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  950.548685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  950.548929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  950.549090] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  950.549343] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  950.549503] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  950.549654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  950.549876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  950.550015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  950.550263] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-6321cea3-fe6e-4300-9416-2206d2ccae8d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  950.550413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  950.550530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  950.550735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  950.550877] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('6321cea3-fe6e-4300-9416-2206d2ccae8d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  950.551179] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a2fb7326a588431a86a87af57b96452d in queue reply_156db6c4256a4c58884afce15e6f5340
[  950.556532] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a2fb7326a588431a86a87af57b96452d
[  950.616441] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  950.616621] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  951.053575] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-6321cea3-fe6e-4300-9416-2206d2ccae8d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  951.054036] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  951.054036] env[60890]: ERROR oslo_messaging.rpc.server 
[  951.845128] env[60890]: INFO networking_nsxv3.common.synchronization [req-092c8aa4-2449-4c2f-8499-2089a162b25c req-b12923d9-259e-44b0-8f0a-976ad6ea968a tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  951.845128] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  951.845128] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  951.845128] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  951.845128] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  951.845128] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  951.845128] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  951.845128] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg e49851e6d6f64159884f204ea7323490 in queue reply_156db6c4256a4c58884afce15e6f5340
[  951.852791] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e49851e6d6f64159884f204ea7323490
[  952.076477] env[60890]: INFO networking_nsxv3.common.synchronization [req-1f436ead-ebe8-47d5-8a9d-9b9b96adf9d4 req-09637044-eb3a-4e34-9438-a58b30b78c23 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  952.079517] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  952.079517] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.079517] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.079517] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.079517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.079517] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  952.079517] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 0b9718f3dbca4a0ba456f8618dbbcb9b in queue reply_156db6c4256a4c58884afce15e6f5340
[  952.086943] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 0b9718f3dbca4a0ba456f8618dbbcb9b
[  952.349683] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.349683] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.349683] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  952.349683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.422908] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.423120] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.423211] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.423475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.423622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.423878] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.424044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.424165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.424382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.424625] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.424780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.424948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.425129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.425270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.425413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.425656] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.425809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.425925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.426130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.426263] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.426495] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.426644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.426757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.427025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.427162] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  952.427389] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ad4d3b2eccbd464c8590ee00203694c9 in queue reply_156db6c4256a4c58884afce15e6f5340
[  952.432368] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ad4d3b2eccbd464c8590ee00203694c9
[  952.581086] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.581247] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.581317] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.581577] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.581681] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.581880] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  952.581984] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  952.582045] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  952.582201] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  952.582276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.582359] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  952.582437] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.930318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  952.930318] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  952.930318] env[60890]: ERROR oslo_messaging.rpc.server 
[  953.497630] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  953.497962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  953.497962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  953.498242] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  953.498356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  953.498607] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  953.498763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  953.498883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  953.499105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  953.499468] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  953.499718] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  953.499915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  953.500181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  953.500328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  953.500480] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  953.500731] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  953.500887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  953.501009] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  953.501218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  953.501355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  953.501598] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  953.502227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  953.502227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  953.502227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  953.502227] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  953.502480] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a96cc7152b4c4e6fa42b9dfe42cdc847 in queue reply_156db6c4256a4c58884afce15e6f5340
[  953.507757] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a96cc7152b4c4e6fa42b9dfe42cdc847
[  954.004122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  954.004662] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  954.004662] env[60890]: ERROR oslo_messaging.rpc.server 
[  954.521567] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg c67f4c70bfb349fb90562bfcb253afd0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  954.533547] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c67f4c70bfb349fb90562bfcb253afd0
[  954.549330] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  954.549542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  954.549671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  954.549947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  954.550177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  954.550489] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  954.550656] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  954.550778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  954.551007] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  954.551256] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  954.551414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  954.551533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  954.551739] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  954.551882] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  954.552046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  954.552302] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  954.552459] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  954.552579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  954.552809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  954.552953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  954.553195] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  954.553351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  954.553472] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  954.553682] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  954.553815] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  954.554045] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2b526292db8f4621be2eb72afc390082 in queue reply_156db6c4256a4c58884afce15e6f5340
[  954.560913] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2b526292db8f4621be2eb72afc390082
[  954.606945] env[60890]: INFO neutron.plugins.ml2.drivers.agent._common_agent [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] NSXv3 Agent Agent loop - iteration:1 completed
[  954.607081] env[60890]: INFO neutron.plugins.ml2.drivers.agent._common_agent [None req-f7775f5d-e50b-414d-8925-beecc2af85d1 None None] NSXv3 Agent Agent loop - iteration:2 started
[  955.056303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  955.056825] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  955.056825] env[60890]: ERROR oslo_messaging.rpc.server 
[  955.595514] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  955.595776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  955.595911] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  955.596258] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  955.596409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  955.596676] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  955.596828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  955.596947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  955.597168] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  955.597409] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  955.597560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  955.597677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  955.597881] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  955.598019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  955.598165] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  955.598409] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  955.598561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  955.598678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  955.598892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  955.599023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  955.599261] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  955.599412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  955.599528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  955.599731] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  955.599872] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  955.600106] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 561a1bab617f4e138748c7623b867e71 in queue reply_156db6c4256a4c58884afce15e6f5340
[  955.605394] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 561a1bab617f4e138748c7623b867e71
[  956.101930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  956.102469] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  956.102469] env[60890]: ERROR oslo_messaging.rpc.server 
[  956.642955] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  956.643237] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  956.643306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  956.643570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  956.643719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  956.643973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  956.644150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  956.644270] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  956.644486] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  956.644729] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  956.644900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  956.645018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  956.645219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  956.645354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  956.645495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  956.645746] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  956.645936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  956.646064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  956.646271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  956.646406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  956.646644] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  956.646797] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  956.646916] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  956.647122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  956.647256] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  956.647485] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c95d5add6bf14227a788a5fd56c3ca00 in queue reply_156db6c4256a4c58884afce15e6f5340
[  956.652883] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c95d5add6bf14227a788a5fd56c3ca00
[  957.077840] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  957.077953] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  957.149740] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  957.150366] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  957.150366] env[60890]: ERROR oslo_messaging.rpc.server 
[  957.259241] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  957.259460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  957.259586] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  957.259861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  957.260025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  957.260288] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  957.260443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  957.260560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  957.260774] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  957.261086] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  957.261184] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  957.261290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  957.261495] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  957.261635] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  957.261783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  957.262034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  957.262188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  957.262305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  957.262509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  957.262642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  957.262906] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  957.263063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  957.263181] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  957.263385] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  957.263516] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  957.263743] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1d728f156319470b898fa372cdac765d in queue reply_156db6c4256a4c58884afce15e6f5340
[  957.270219] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1d728f156319470b898fa372cdac765d
[  957.766194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  957.766622] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  957.766622] env[60890]: ERROR oslo_messaging.rpc.server 
[  958.423978] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  958.424222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  958.424355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  958.424627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  958.424785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  958.425075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  958.425244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  958.425367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  958.425612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  958.425828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  958.425986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  958.426106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  958.426314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  958.426457] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  958.426609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  958.426863] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  958.427024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  958.427155] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  958.427383] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  958.427525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  958.427769] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  958.427929] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  958.428070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  958.428287] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  958.428417] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  958.428653] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 52bd7d30f2bf4b8bb05078b011b569bf in queue reply_156db6c4256a4c58884afce15e6f5340
[  958.435200] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 52bd7d30f2bf4b8bb05078b011b569bf
[  958.931826] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  958.932393] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  958.932393] env[60890]: ERROR oslo_messaging.rpc.server 
[  959.471104] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  959.471225] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  959.471350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  959.471614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  959.471765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  959.472024] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  959.472188] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  959.472310] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  959.472529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  959.472776] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  959.473086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  959.473230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  959.473466] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  959.473618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  959.473771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  959.474033] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  959.474198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  959.474319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  959.474632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  959.474696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  959.475018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  959.475231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  959.475359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  959.475581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  959.475746] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  959.476030] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ceab9c652fd84d3d828805274971e71c in queue reply_156db6c4256a4c58884afce15e6f5340
[  959.481611] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ceab9c652fd84d3d828805274971e71c
[  959.979333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  959.980429] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  959.980429] env[60890]: ERROR oslo_messaging.rpc.server 
[  960.515206] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  960.515206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  960.515394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  960.515604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  960.515756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  960.516071] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  960.516246] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  960.516369] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  960.516601] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  960.516861] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  960.517026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  960.517145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  960.517356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  960.517504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  960.517655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  960.517908] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  960.518068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  960.518187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  960.518397] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  960.518535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  960.518777] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  960.518934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  960.519054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  960.519261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  960.519396] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  960.519622] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 09ba79ab81644a33ab595a5b1600c564 in queue reply_156db6c4256a4c58884afce15e6f5340
[  960.524596] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 09ba79ab81644a33ab595a5b1600c564
[  961.021196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  961.021693] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  961.021693] env[60890]: ERROR oslo_messaging.rpc.server 
[  961.563337] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  961.563553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  961.563685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  961.563968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  961.564136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  961.564407] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  961.564567] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  961.564686] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  961.564943] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  961.565196] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  961.565356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  961.565475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  961.565687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  961.565835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  961.565982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  961.566235] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  961.566391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  961.566510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  961.566735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  961.566877] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  961.567165] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-ca601dbb-baac-47fb-9260-0a887d117a73 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  961.567276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  961.567395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  961.567602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  961.567739] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('ca601dbb-baac-47fb-9260-0a887d117a73',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  961.567978] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5e9ed6f67879485683ef316fc29db8ca in queue reply_156db6c4256a4c58884afce15e6f5340
[  961.573182] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5e9ed6f67879485683ef316fc29db8ca
[  962.070208] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-ca601dbb-baac-47fb-9260-0a887d117a73" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  962.070729] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  962.070729] env[60890]: ERROR oslo_messaging.rpc.server 
[  962.079208] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  962.079346] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  962.834494] env[60890]: INFO networking_nsxv3.common.synchronization [req-1f436ead-ebe8-47d5-8a9d-9b9b96adf9d4 req-7aba4981-842f-4fc8-a05c-d8aec2d2a5e1 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  962.835059] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  962.835338] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  962.835465] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  962.835527] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  962.835715] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  962.835799] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  962.835950] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  962.836063] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  962.836123] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  962.836269] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  962.836403] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  962.836488] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  962.836545] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  962.836675] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  962.836748] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  962.836831] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  962.836998] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3d9d2e60434b41e0ba12be1a62a91ac6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  962.844206] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3d9d2e60434b41e0ba12be1a62a91ac6
[  963.339321] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  963.339695] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  963.339695] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  963.339814] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  963.339864] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  963.340044] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  963.340148] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  963.340203] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  963.340357] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  963.340436] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  963.340522] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  963.340631] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.053246] env[60890]: INFO networking_nsxv3.common.synchronization [req-43e08119-e127-46b3-8420-a8cc5d247708 req-b2f3a66b-1717-47e0-bad2-3ed6a03a22d0 tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  966.053246] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  966.053246] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.053246] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.053246] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.053246] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.053246] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  966.053246] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 292ff73801f34e98bbb92fbd242cef72 in queue reply_156db6c4256a4c58884afce15e6f5340
[  966.065580] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 292ff73801f34e98bbb92fbd242cef72
[  966.300802] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.300802] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.300802] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.300802] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.300802] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.300802] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.301637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.301637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.301637] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.301637] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  966.301777] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ab552371428d4a8faadca36662735205 in queue reply_156db6c4256a4c58884afce15e6f5340
[  966.307215] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ab552371428d4a8faadca36662735205
[  966.558804] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.558946] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.558988] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.559279] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.559384] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.559561] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  966.559661] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  966.559725] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  966.559863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  966.559941] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.560036] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  966.560127] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.804218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  966.804773] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  966.804773] env[60890]: ERROR oslo_messaging.rpc.server 
[  967.342098] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  967.342348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  967.342426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  967.342687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  967.342835] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  967.343091] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  967.343245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  967.343363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  967.343572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  967.343813] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  967.343968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  967.344100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  967.344305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  967.344445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  967.344589] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  967.344851] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  967.345008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  967.345131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  967.345340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  967.345475] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  967.345708] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  967.345879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  967.346006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  967.346214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  967.346346] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  967.346656] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg aa2440157f024e699de82b26798403e4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  967.351858] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg aa2440157f024e699de82b26798403e4
[  967.848776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  967.849336] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  967.849336] env[60890]: ERROR oslo_messaging.rpc.server 
[  968.386715] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  968.386715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  968.386715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  968.387111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  968.387111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  968.387273] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  968.387412] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  968.387531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  968.387751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  968.388015] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  968.388177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  968.388297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  968.388501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  968.388642] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  968.388787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  968.389045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  968.389196] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  968.389313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  968.389604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  968.389757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  968.390010] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  968.390164] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  968.390387] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  968.390541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  968.390679] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  968.390913] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a3c6375c1b0f4336b4c9442600cd7841 in queue reply_156db6c4256a4c58884afce15e6f5340
[  968.396382] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a3c6375c1b0f4336b4c9442600cd7841
[  968.893990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  968.893990] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  968.893990] env[60890]: ERROR oslo_messaging.rpc.server 
[  969.431555] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  969.431862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  969.431923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  969.432186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  969.432338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  969.432601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  969.432789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  969.432909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  969.433136] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  969.433383] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  969.433541] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  969.433663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  969.433875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  969.434018] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  969.434167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  969.434419] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  969.434578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  969.434696] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  969.434944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  969.435086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  969.435326] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  969.435482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  969.435602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  969.435851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  969.435954] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  969.436188] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3da15d2e63fb4f7fab5b3e857ebd3a60 in queue reply_156db6c4256a4c58884afce15e6f5340
[  969.441332] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3da15d2e63fb4f7fab5b3e857ebd3a60
[  969.939031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  969.939247] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  969.939247] env[60890]: ERROR oslo_messaging.rpc.server 
[  970.475348] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  970.475688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  970.475688] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  970.475947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  970.476107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  970.476365] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  970.476523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  970.476644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  970.476866] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  970.477116] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  970.477307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  970.477432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  970.477647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  970.477791] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  970.477945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  970.478198] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  970.478355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  970.478476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  970.478690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  970.478828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  970.479069] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  970.479224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  970.479342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  970.479554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  970.479692] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  970.479924] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 348a7c6d98ca4e2ea13c378fa2fa2dc1 in queue reply_156db6c4256a4c58884afce15e6f5340
[  970.485188] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 348a7c6d98ca4e2ea13c378fa2fa2dc1
[  970.981889] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  970.982422] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  970.982422] env[60890]: ERROR oslo_messaging.rpc.server 
[  971.051385] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  971.051493] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  971.522389] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  971.523014] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  971.523307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  971.523704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  971.524026] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  971.524433] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  971.524754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  971.525028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  971.525392] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  971.525779] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  971.526107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  971.526366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  971.526703] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  971.527004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  971.527290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  971.527669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  971.528022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  971.528408] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  971.528759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  971.529035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  971.529401] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  971.529717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  971.529970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  971.530334] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  971.530626] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  971.530986] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5137671a56ad43a5bd2ad33d313af590 in queue reply_156db6c4256a4c58884afce15e6f5340
[  971.536178] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5137671a56ad43a5bd2ad33d313af590
[  972.033608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  972.034136] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  972.034136] env[60890]: ERROR oslo_messaging.rpc.server 
[  972.567245] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  972.567245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  972.567245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  972.567245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  972.567245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  972.567245] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  972.567245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  972.567245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  972.567245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  972.567245] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  972.567910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  972.567910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  972.567910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  972.567910] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  972.568057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  972.568258] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  972.568416] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  972.568533] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  972.568747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  972.568915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  972.569166] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  972.569322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  972.569434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  972.569639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  972.569771] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  972.569999] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f65655567efd4aada94055d5d9391b34 in queue reply_156db6c4256a4c58884afce15e6f5340
[  972.574700] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f65655567efd4aada94055d5d9391b34
[  973.072423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  973.072934] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  973.072934] env[60890]: ERROR oslo_messaging.rpc.server 
[  973.617908] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  973.618234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  973.618294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  973.618526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  973.618677] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  973.618942] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  973.619105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  973.619227] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  973.619450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  973.619744] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  973.619936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  973.620106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  973.620338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  973.620485] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  973.620638] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  973.620894] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  973.621054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  973.621174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  973.621388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  973.621525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  973.621766] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  973.621922] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  973.622142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  973.622365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  973.622503] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  973.622756] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 40761dc652cf4ea9aa574c86cb593dae in queue reply_156db6c4256a4c58884afce15e6f5340
[  973.629240] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 40761dc652cf4ea9aa574c86cb593dae
[  974.125492] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  974.126008] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  974.126008] env[60890]: ERROR oslo_messaging.rpc.server 
[  974.660553] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  974.660828] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  974.660865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  974.661128] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  974.661267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  974.661519] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  974.661670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  974.661784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  974.662024] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  974.662264] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  974.662413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  974.662527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  974.662949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  974.662949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  974.663054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  974.663275] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  974.663425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  974.663539] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  974.663741] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  974.663869] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  974.664117] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  974.664264] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  974.664378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  974.664578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  974.664705] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  974.664981] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 546e7a80de4646e5ae0219696e591223 in queue reply_156db6c4256a4c58884afce15e6f5340
[  974.669722] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 546e7a80de4646e5ae0219696e591223
[  975.166935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  975.167483] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  975.167483] env[60890]: ERROR oslo_messaging.rpc.server 
[  975.702687] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.703087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.703256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.703678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.703962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  975.704389] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-870 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.704662] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.704915] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.705290] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.705696] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.705966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.706193] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.706535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.706793] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  975.707096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-870" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  975.707488] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.707749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.707967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.708321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.708562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  975.708957] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-801d300e-749b-4db4-a136-e35215463cbe taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.709214] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.709438] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.709776] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.710024] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('801d300e-749b-4db4-a136-e35215463cbe',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  975.710393] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg dfd0e0a0d75c4180b49d915502af439f in queue reply_156db6c4256a4c58884afce15e6f5340
[  975.715512] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dfd0e0a0d75c4180b49d915502af439f
[  975.809649] env[60890]: INFO networking_nsxv3.common.synchronization [req-7381106d-a918-43c5-8165-e19dccad484d req-d4c633f8-deab-4771-a75c-15faad947ba0 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  975.810189] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  975.810523] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.810651] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.810718] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.810914] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.811004] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  975.811172] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.811282] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.811343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.811500] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.811642] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  975.811735] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  975.811793] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  975.811936] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  975.812048] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  975.812144] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  975.812330] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 239773ca31cb4bf3859652ba8c391650 in queue reply_156db6c4256a4c58884afce15e6f5340
[  975.820689] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 239773ca31cb4bf3859652ba8c391650
[  976.092238] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.092441] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.092571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.092879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.093031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.093289] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.093447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.093565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.093788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.094045] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.094203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.094321] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.094528] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.094666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.094813] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.095095] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.095256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.095373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.095581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.095715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.095976] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.096147] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.096265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.096535] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.096670] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  976.096930] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bad378da13134922bc715dc6feabf504 in queue reply_156db6c4256a4c58884afce15e6f5340
[  976.101708] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bad378da13134922bc715dc6feabf504
[  976.213093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-801d300e-749b-4db4-a136-e35215463cbe" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  976.213603] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  976.213603] env[60890]: ERROR oslo_messaging.rpc.server 
[  976.318105] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.318306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.318306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.318530] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.318620] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.318812] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.318913] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.318976] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.319118] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.319192] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.319276] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  976.319345] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.598753] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  976.599253] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  976.599253] env[60890]: ERROR oslo_messaging.rpc.server 
[  976.949614] env[60890]: INFO networking_nsxv3.common.synchronization [req-43e08119-e127-46b3-8420-a8cc5d247708 req-a54431ac-a153-4ffe-9f06-93802d26656b tempest-DeleteServersTestJSON-290217981 tempest-DeleteServersTestJSON-290217981-project-member] Enqueued Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  976.950106] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 with Priority: HIGHEST for action security_group_members
[  976.950426] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.950550] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.950618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.950813] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.950899] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.951047] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.951142] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.951203] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.951345] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.951481] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  976.951569] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  976.951628] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  976.951784] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  976.951832] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  976.951918] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('e7e4f653-60b4-400c-bc36-6f4f1bc63f83',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  976.952138] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4e33906b29c04ad8b8d29ff1609addf7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  976.961129] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4e33906b29c04ad8b8d29ff1609addf7
[  977.136603] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  977.136815] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  977.136946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  977.137222] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  977.137374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.138037] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  977.138242] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  977.138371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  977.138607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  977.138858] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  977.139030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  977.139177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  977.139465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  977.139622] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.139783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.140061] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  977.140226] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  977.140374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  977.140597] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  977.140735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.140976] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  977.141137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  977.141256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  977.141465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  977.141640] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  977.141917] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 99ff2f39d3814ee0aed24aec1ebfd134 in queue reply_156db6c4256a4c58884afce15e6f5340
[  977.146649] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 99ff2f39d3814ee0aed24aec1ebfd134
[  977.453997] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  977.454146] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  977.454223] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  977.454431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  977.454525] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.454713] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  977.454813] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  977.454910] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  977.455067] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  977.455145] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.455231] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: e7e4f653-60b4-400c-bc36-6f4f1bc63f83 already deleted.
[  977.455308] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-e7e4f653-60b4-400c-bc36-6f4f1bc63f83" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.644337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  977.644891] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  977.644891] env[60890]: ERROR oslo_messaging.rpc.server 
[  978.184938] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  978.185244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  978.185289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  978.185536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  978.185683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  978.185944] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  978.186100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  978.186231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  978.186456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  978.186698] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  978.186855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  978.186975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  978.187183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  978.187324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  978.187470] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  978.187715] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  978.187870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  978.187989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  978.188238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  978.188380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  978.188620] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  978.188775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  978.188892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  978.189096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  978.189229] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  978.189453] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 276e140bef0a44e792f96d94021ca1f6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  978.194329] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 276e140bef0a44e792f96d94021ca1f6
[  978.691422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  978.691890] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  978.691890] env[60890]: ERROR oslo_messaging.rpc.server 
[  979.228167] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  979.228461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  979.228507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  979.228782] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  979.228933] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  979.229194] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  979.229352] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  979.229473] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  979.229693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  979.229939] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  979.230274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  979.230274] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  979.230574] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  979.230658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  979.230814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  979.231062] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  979.231219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  979.231342] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  979.231593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  979.231732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  979.231973] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  979.232200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  979.232328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  979.232594] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  979.232741] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  979.232970] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg adfbf446c0944c9ba3231b4c93901af7 in queue reply_156db6c4256a4c58884afce15e6f5340
[  979.238164] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg adfbf446c0944c9ba3231b4c93901af7
[  979.735690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  979.736281] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  979.736281] env[60890]: ERROR oslo_messaging.rpc.server 
[  980.543363] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  980.543671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  980.543714] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  980.543956] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  980.544122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  980.544380] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  980.544536] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  980.544655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  980.544894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  980.545146] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  980.545303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  980.545421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  980.545629] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  980.545770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  980.545919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  980.546176] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  980.546330] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  980.546448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  980.546655] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  980.546788] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  980.547024] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  980.547185] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  980.547304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  980.547506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  980.547637] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  980.547861] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2a1170fb4ff544c2850832ce302f0d59 in queue reply_156db6c4256a4c58884afce15e6f5340
[  980.552896] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2a1170fb4ff544c2850832ce302f0d59
[  981.049779] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  981.050289] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  981.050289] env[60890]: ERROR oslo_messaging.rpc.server 
[  981.600063] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  981.600063] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  981.600063] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  981.600063] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  981.600855] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  981.600855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  981.600855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  981.600855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  981.601008] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  981.601208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  981.601364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  981.601482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  981.601693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  981.601829] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  981.602082] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9665c1abb08e411b9b921f5a916242ec in queue reply_156db6c4256a4c58884afce15e6f5340
[  981.607570] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9665c1abb08e411b9b921f5a916242ec
[  981.950724] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  981.950881] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  982.104083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  982.104601] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  982.104601] env[60890]: ERROR oslo_messaging.rpc.server 
[  982.645577] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  982.645893] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  982.645935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  982.646204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  982.646353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  982.646610] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  982.646768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  982.646890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  982.647109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  982.647420] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  982.647595] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  982.647716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  982.647931] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  982.648095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  982.648256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  982.648503] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  982.648658] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  982.648778] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  982.648987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  982.649124] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  982.649361] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  982.649517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  982.649664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  982.649845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  982.649982] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  982.650205] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 614e6af9406340adaad6a500d02fb2cb in queue reply_156db6c4256a4c58884afce15e6f5340
[  982.655071] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 614e6af9406340adaad6a500d02fb2cb
[  983.151957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  983.152513] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  983.152513] env[60890]: ERROR oslo_messaging.rpc.server 
[  983.691434] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  983.691732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  983.691732] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  983.691967] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  983.692134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  983.692389] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  983.692545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  983.692664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  983.692880] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  983.693123] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  983.693282] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  983.693404] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  983.693611] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  983.693755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  983.693906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  983.694158] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  983.694315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  983.694435] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  983.694644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  983.694780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  983.695029] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  983.695186] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  983.695306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  983.695513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  983.695645] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  983.695895] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2541b46729f84d08a7ef8d53f0aca3b8 in queue reply_156db6c4256a4c58884afce15e6f5340
[  983.702381] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2541b46729f84d08a7ef8d53f0aca3b8
[  984.198454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  984.198977] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  984.198977] env[60890]: ERROR oslo_messaging.rpc.server 
[  984.524275] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5cc6492b80b8448c9e6af2129e3928ec in queue reply_156db6c4256a4c58884afce15e6f5340
[  984.538978] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5cc6492b80b8448c9e6af2129e3928ec
[  984.734501] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  984.734856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  984.735068] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  984.735353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  984.735501] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  984.735763] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  984.735923] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  984.736057] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  984.736276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  984.736565] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  984.736675] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  984.736794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  984.737004] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  984.737148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  984.737297] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  984.737545] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  984.737698] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  984.737814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  984.738022] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  984.738158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  984.738394] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  984.738550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  984.738668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  984.738872] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  984.739006] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  984.739233] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg a8ed6b6bdfaa40ddb13f59623e2cfffa in queue reply_156db6c4256a4c58884afce15e6f5340
[  984.744707] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a8ed6b6bdfaa40ddb13f59623e2cfffa
[  985.240938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  985.241426] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  985.241426] env[60890]: ERROR oslo_messaging.rpc.server 
[  985.777115] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  985.777476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  985.777476] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  985.777848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  985.777981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  985.778281] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-57 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  985.778477] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  985.778631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  985.778888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  985.779183] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  985.779375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  985.779525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  985.779769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  985.779949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  985.780175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-57" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  985.780470] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  985.780666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  985.780818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  985.781064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  985.781234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  985.781514] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  985.781707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  985.781945] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  985.782354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  985.782584] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('97b5d1d3-b592-4e55-a1dd-e09fef7190e7',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  985.782849] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b2b76c1b6da04c2a9afd94c3ef459f32 in queue reply_156db6c4256a4c58884afce15e6f5340
[  985.787690] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b2b76c1b6da04c2a9afd94c3ef459f32
[  986.284749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-97b5d1d3-b592-4e55-a1dd-e09fef7190e7" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  986.285389] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  986.285389] env[60890]: ERROR oslo_messaging.rpc.server 
[  986.952305] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[  986.952606] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[  987.155838] env[60890]: INFO networking_nsxv3.common.synchronization [req-7381106d-a918-43c5-8165-e19dccad484d req-aa96d16a-9585-42f2-8da2-9b5a72cc03e1 tempest-ServersTestJSON-2010463638 tempest-ServersTestJSON-2010463638-project-member] Enqueued Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  987.155838] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c697a421-4b6e-450e-8c23-f5f9277f9219 with Priority: HIGHEST for action security_group_members
[  987.156100] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  987.156186] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  987.156258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  987.156476] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  987.156586] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  987.156752] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-c697a421-4b6e-450e-8c23-f5f9277f9219 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  987.156856] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  987.156921] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  987.157076] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  987.157223] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  987.157301] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  987.157363] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  987.157502] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  987.157581] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  987.157672] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('c697a421-4b6e-450e-8c23-f5f9277f9219',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  987.157848] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d6985ce52a4d4e4ab857c3ebf8348cb4 in queue reply_156db6c4256a4c58884afce15e6f5340
[  987.169247] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d6985ce52a4d4e4ab857c3ebf8348cb4
[  987.662157] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  987.662303] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  987.662375] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  987.662588] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  987.662682] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  987.662891] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  987.663009] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  987.663090] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  987.663219] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  987.663298] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  987.663383] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: c697a421-4b6e-450e-8c23-f5f9277f9219 already deleted.
[  987.663461] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-c697a421-4b6e-450e-8c23-f5f9277f9219" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  989.615109] env[60890]: INFO networking_nsxv3.common.synchronization [None req-49bb3a53-d5ea-44db-b39f-9755ac050769 None None] Enqueued Resource ID: 5c4da857-8bfb-4d6b-90a6-733dfeeaeee0 with Priority: HIGHEST for action port
[  989.615723] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 5c4da857-8bfb-4d6b-90a6-733dfeeaeee0 with Priority: HIGHEST for action port
[  989.616104] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  989.616233] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  989.616299] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  989.616528] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  989.616615] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  989.616768] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-5c4da857-8bfb-4d6b-90a6-733dfeeaeee0 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  989.616866] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-5c4da857-8bfb-4d6b-90a6-733dfeeaeee0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  989.616928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-5c4da857-8bfb-4d6b-90a6-733dfeeaeee0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  989.617216] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-5c4da857-8bfb-4d6b-90a6-733dfeeaeee0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  989.617296] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5c4da857-8bfb-4d6b-90a6-733dfeeaeee0',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  989.617475] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ff6f5d8fa7074225bbd776bb84e16b3a in queue reply_156db6c4256a4c58884afce15e6f5340
[  989.712493] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ff6f5d8fa7074225bbd776bb84e16b3a
[  990.120207] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-5c4da857-8bfb-4d6b-90a6-733dfeeaeee0" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  990.120463] env[60890]: Traceback (most recent call last):
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  990.120505] env[60890]:     timer()
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  990.120505] env[60890]:     cb(*args, **kw)
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  990.120505] env[60890]:     result = function(*args, **kwargs)
[  990.120505] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  990.120505] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  990.120505] env[60890]:     return method(*args, **kwargs)
[  990.120505] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  990.120505] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  990.120505] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  990.120505] env[60890]:     result = self.transport._send(
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  990.120505] env[60890]:     return self._driver.send(target, ctxt, message,
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  990.120505] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  990.120505] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  990.120505] env[60890]:     raise result
[  990.121115] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  990.121115] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  991.612498] env[60890]: INFO networking_nsxv3.common.synchronization [None req-d3516976-7b1a-4f0b-afb0-1b212e468d4a None None] Enqueued Resource ID: 0e1fa487-8a32-4779-8098-2334618e5fa1 with Priority: HIGHEST for action port
[  991.612805] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0e1fa487-8a32-4779-8098-2334618e5fa1 with Priority: HIGHEST for action port
[  991.613528] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  991.613661] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  991.613731] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  991.613934] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  991.614020] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  991.614180] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-0e1fa487-8a32-4779-8098-2334618e5fa1 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  991.614276] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-0e1fa487-8a32-4779-8098-2334618e5fa1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  991.614339] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-0e1fa487-8a32-4779-8098-2334618e5fa1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  991.614570] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-0e1fa487-8a32-4779-8098-2334618e5fa1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  991.614650] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0e1fa487-8a32-4779-8098-2334618e5fa1',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  991.614844] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8fdc5ee97d8647a39222c63e2f9c19c6 in queue reply_156db6c4256a4c58884afce15e6f5340
[  991.620599] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8fdc5ee97d8647a39222c63e2f9c19c6
[  992.117212] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-0e1fa487-8a32-4779-8098-2334618e5fa1" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  992.117212] env[60890]: Traceback (most recent call last):
[  992.117212] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[  992.117212] env[60890]:     timer()
[  992.117212] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[  992.117212] env[60890]:     cb(*args, **kw)
[  992.117212] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[  992.117212] env[60890]:     result = function(*args, **kwargs)
[  992.117554] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[  992.117554] env[60890]:     port: dict = self.rpc.get_port(os_id)
[  992.117554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  992.117554] env[60890]:     return method(*args, **kwargs)
[  992.117554] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  992.117554] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  992.117554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  992.117554] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[  992.117554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  992.117554] env[60890]:     result = self.transport._send(
[  992.117554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  992.117554] env[60890]:     return self._driver.send(target, ctxt, message,
[  992.117554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  992.117554] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  992.117554] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  992.117554] env[60890]:     raise result
[  992.117554] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  992.117554] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  995.072782] env[60890]: INFO networking_nsxv3.common.synchronization [req-37a0142c-f247-4259-a802-939d01e1e7f7 req-35f8b555-cb90-44a9-af63-1a8345173844 tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[  995.073306] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[  995.073622] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.073743] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.073809] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.074012] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.074118] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.074273] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.074368] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.074428] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.074614] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.074753] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.074903] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.074960] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.075088] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.075164] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.075251] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  995.075420] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d6624a07acc34a4996e58dbb647facb0 in queue reply_156db6c4256a4c58884afce15e6f5340
[  995.084048] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d6624a07acc34a4996e58dbb647facb0
[  995.335217] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.335433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.335555] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.335836] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.335992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.336277] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.336429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.336861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.336861] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.337018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.337243] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.337307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.337488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.337632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.337783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.338033] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.338189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.338308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.338514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.338651] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.338890] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.339046] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.339166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.339431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.339655] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  995.339907] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8ccc1b05f4d5465ea01fd191ba649117 in queue reply_156db6c4256a4c58884afce15e6f5340
[  995.345169] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8ccc1b05f4d5465ea01fd191ba649117
[  995.577170] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.577484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.577484] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.577619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.577683] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.577876] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  995.577975] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  995.578031] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  995.578180] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  995.578258] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.578340] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[  995.578440] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.842366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  995.842771] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  995.842771] env[60890]: ERROR oslo_messaging.rpc.server 
[  996.380744] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  996.381051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  996.381103] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  996.381348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  996.381505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  996.381774] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  996.381936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  996.382093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  996.382326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  996.382578] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  996.382744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  996.382864] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  996.383075] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  996.383219] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  996.383374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  996.383626] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  996.383785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  996.383905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  996.384143] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  996.384284] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  996.384527] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  996.384685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  996.384806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  996.385060] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  996.385204] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  996.385435] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ba05f6572cc943939477040932439c89 in queue reply_156db6c4256a4c58884afce15e6f5340
[  996.390810] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ba05f6572cc943939477040932439c89
[  996.887212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  996.887900] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  996.887900] env[60890]: ERROR oslo_messaging.rpc.server 
[  997.479556] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  997.479819] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  997.479899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  997.480253] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  997.480406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  997.480683] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  997.480862] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  997.480988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  997.481205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  997.481450] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  997.481605] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  997.481724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  997.481930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  997.482093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  997.482276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  997.482533] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  997.482687] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  997.482823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  997.483089] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  997.483229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  997.483470] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  997.483667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  997.483796] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  997.484015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  997.484236] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  997.484395] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f1ae7e422edd481b93b386ef5f554eaa in queue reply_156db6c4256a4c58884afce15e6f5340
[  997.490579] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f1ae7e422edd481b93b386ef5f554eaa
[  997.986428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  997.986926] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  997.986926] env[60890]: ERROR oslo_messaging.rpc.server 
[  998.527195] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  998.527550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  998.527550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  998.527783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  998.527928] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  998.528210] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  998.528393] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  998.528510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  998.528724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  998.528966] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  998.529137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  998.529324] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  998.529546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  998.529693] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  998.529848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  998.530102] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  998.530256] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  998.530375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  998.530578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  998.530713] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  998.530952] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  998.531110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  998.531221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  998.531428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  998.531555] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  998.531780] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f13ca40703544193b267ba1102b38490 in queue reply_156db6c4256a4c58884afce15e6f5340
[  998.536706] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f13ca40703544193b267ba1102b38490
[  999.033515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  999.034048] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[  999.034048] env[60890]: ERROR oslo_messaging.rpc.server 
[  999.575588] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  999.575924] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  999.575982] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  999.576271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  999.576426] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  999.576693] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  999.576857] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  999.576984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  999.577203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  999.577448] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  999.577604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  999.577725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  999.577934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  999.578078] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  999.578230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  999.578483] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  999.578639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  999.578759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  999.578968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  999.579107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[  999.579347] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[  999.579505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[  999.579626] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[  999.579832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[  999.579966] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[  999.580217] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 83198ac2f2d3482d9aad686a89d7c6db in queue reply_156db6c4256a4c58884afce15e6f5340
[  999.585006] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 83198ac2f2d3482d9aad686a89d7c6db
[ 1000.074267] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1000.074463] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1000.082011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1000.082510] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1000.082510] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1000.345605] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 97 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1000.345795] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1000.345869] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1000.346201] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1000.346292] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1000.347275] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock all taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 100 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1000.347397] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1000.347473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1000.347619] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1000.347728] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_ports_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1000.347921] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg eaf0306442274fd882cba472ad329ffe in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1000.354518] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg eaf0306442274fd882cba472ad329ffe
[ 1000.618941] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1000.619245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1000.619313] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1000.619531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1000.619681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1000.619941] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1000.620127] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1000.620250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1000.620464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1000.620711] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1000.620867] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1000.620986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1000.621190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1000.621340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1000.621489] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1000.621734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1000.621891] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1000.622010] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1000.622217] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1000.622349] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1000.622584] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1000.622744] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1000.622947] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1000.623173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1000.623311] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1000.623535] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 77c302b9b37f40ba9def5e7f8a28dfba in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1000.628702] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 77c302b9b37f40ba9def5e7f8a28dfba
[ 1000.850363] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_security_groups_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1000.850557] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 3fa5bea1167648e79e158f7366583d7b in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1000.857640] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3fa5bea1167648e79e158f7366583d7b
[ 1001.125883] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1001.126416] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1001.126416] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1001.352794] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_qos_policies_with_revisions called with arguments (1000, 0) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1001.353130] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d5d72616577b4e518bb569d4b5d6ed43 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1001.360333] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d5d72616577b4e518bb569d4b5d6ed43
[ 1001.664895] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1001.665220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1001.665261] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1001.665496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1001.665641] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1001.665912] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1001.666158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1001.666298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1001.666518] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1001.666764] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1001.666983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1001.667122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1001.667359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1001.667507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1001.667654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1001.667912] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1001.668082] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1001.668203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1001.668409] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1001.668619] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1001.668874] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1001.669037] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1001.669157] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1001.669365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1001.669504] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1001.669725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 652eff87ecee4c209642b7d6634e9508 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1001.674559] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 652eff87ecee4c209642b7d6634e9508
[ 1001.855699] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Inventory metadata is going to be refreshed.
[ 1001.855935] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Segment.
[ 1002.171822] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1002.172357] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1002.172357] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1002.595841] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 118 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1002.596088] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1002.596088] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1002.596283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1002.610037] env[60890]: WARNING networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider [-] Duplicate resource with OS_ID: 992 ID: cl2-zone-992
[ 1002.610527] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1002.611381] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:SegmentPort.
[ 1002.643821] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock SegmentPort taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 120 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1002.644055] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1002.644055] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1002.644231] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1002.657160] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "SegmentPort" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1002.657774] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:SegmentPort Is:96.
[ 1002.657885] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:SegmentPort Are:{'519bac90-d40c-494c-8eb9-6c53e239f10a', '32332759-ca87-4ef6-955a-325ad700200d', '03fdd0ff-044d-4309-905b-d51a3bab1b81', '55f117f0-ba76-491f-b5c5-7441d3ff7787', '9435dc4a-ead0-4afd-a099-668739ae29f4', '66d98031-6842-491c-9961-c1b4d0fd5d53', '0f875a82-d52b-4617-86ef-09c6a8aeccbb', '2126120d-06ba-46a5-a74c-77f3b18345ab', '17b157f7-bf15-4a5c-a271-9af72764322d', '474946e7-99a3-4290-959e-f2dc1645b999', '4fb6c40c-bf99-4b55-9f6b-ca451ded5287', '7e152039-3f45-45d1-b2a1-c7acb717f68b', '1ca72486-d800-43c5-907b-725bb958b3a3', '2a1bc88a-7641-4017-874c-f15ee8ec3621', 'd357f4a7-7fe1-4677-a32d-b2a6c5780592', '99c860bb-14da-4e95-b94e-d77d15224b7a', 'fb2c7956-257f-4d1d-8dbb-83b5350798c3', 'c4585efa-40da-4b2f-81a8-523efec14826', '7e3eabc0-1383-412e-a875-c4e73daf6808', 'f599de71-cb96-446c-b853-bfcdc671260e', '31125a4c-2800-482d-8420-53a798a29c02', '0cb4488c-6859-4b6c-b322-dc290dbf0f8f', '3e63d897-dbf9-4401-a6d7-1650e7c0ecdf', '62510d77-c23b-4044-a0f1-0f38a92caf2c', '11ab945e-c9e0-4540-8ddf-a57fc4983384', '0ecbd240-2457-42ca-b168-e67fa16f56b3', '156f2622-5bca-4370-8d7a-06130b978cf7', '6f69240a-5122-4e7e-87af-f288a47b5ea9', 'e758e5fd-8995-41a7-9fe7-7ce7894fb7aa', '1e891f4b-00a4-481c-a00c-0320f9e75e62', 'd43c8245-514d-4c94-80b7-c12dfe9a6434', '088ac320-7bf8-4ebf-9318-5c12a3de2e62', '78c543de-6fc1-464e-b0a5-b0db23f19be3', 'ea8f9597-af66-4849-94da-7357f4969707', '889eac8e-299a-4c2b-a8e4-819f8f54a261', '08b26599-acd3-4729-9ae4-7cef241ebca7', '046ad10e-2286-420f-b6f3-0d598b3fcc03', 'bc6fc679-542b-4d2b-838e-70db009ad31a', '64487042-dc1f-4bbb-8b1c-0ff06c270b8d', '45968fc0-6e74-43f7-862b-4043c3249370', '39825d23-3b58-4cdb-8fa7-5a9ff493c789', '2eea5bcc-a0a3-4963-a879-5bf0c1c38f60', '2763deba-ab9e-48b4-b5a8-f7082a993850', 'e9ad7926-1d42-4ff9-aab3-d520de5ae40e', '33359e95-dbc3-4199-9367-e6bd0129468e', '32f3a7a3-308f-4fc7-8898-ed5ad6161587', '88452c84-34f9-49bc-aa27-a6d503e47fa4', 'e1979b45-d8d8-4076-a153-fa64ba78b7c0', 'ace6f305-6029-421e-89d3-bcca9fd2264a', '06f70962-82ab-4995-b7ca-a341783a2c6b', 'ac9ba9d7-c9f6-482f-9cbe-4ac9659c3194', '2da5eafe-fcd1-44dd-b2aa-c1bc16ff3d23', '34dd8f0f-dd08-44fa-9865-37de198ce337', '02696c3f-3dd0-499c-9baa-8a8cf21253eb', 'a385dc0d-17c5-49e9-9685-8285fc885a67', '1b4cdb06-1445-4694-a8a7-25090c7c8982', '74ea2df8-d554-4541-bfb6-8a8fad4756ea', '28285b72-9ad5-4703-950f-d20766e1592d', 'c4c889cc-a303-49e8-b5da-c71c873abb02', '885d71ab-3cf8-44a1-98fe-fedf69f9f021', 'e798c106-4155-4e94-90dc-5591d8705c9e', 'f5808565-bb46-4620-9c27-c24fe7a3d1b6', 'a161c3dd-1d1b-49c2-816f-07c0cc3fe929', '6865e1e1-9000-4b73-b41e-4a4753755582', '271514e3-602e-4224-bfb9-99943b868420', '9674a2da-6ae2-430a-80a2-67bab8177b7e', '8e1ea150-2eec-4dac-80e6-7d1daeaf351d', 'dcde1b76-b76a-4b6b-bbc8-099834069781', '925b62f2-60ee-49b1-974b-66ad3d724498', 'e9d5bf46-3b2c-4088-a6bf-07b65f970ae4', 'f467c1f7-4700-4387-bb2a-249f697bbb6c', '7fcb3b5c-ae38-468f-aecc-8e3293f03e66', 'c4d845cb-64ec-4e2f-87a3-29dcdf4f4b2b', 'c8e81773-32ca-437e-9971-50cef2a264ea', 'd73acfa0-7f7a-4f3e-98ae-e9e2b940b462', 'a367d71c-4063-4a70-9c6c-4f60da50879e', '3903c4b9-6919-4b40-9189-942790370190', 'f1f40c6a-f2c1-4a74-94e2-cdf2e32e24cd', '6040c7d6-38fb-48bc-9161-2b810f0ffb1a', 'fc8312bb-e44e-4d35-b825-f0e74adf2260', 'fa51a02a-4b3e-479b-b703-759a3babfeb1', 'f1446fa4-56d0-44ca-8222-7ed499af9ee1', 'c9f3dd72-2950-4d13-9d1f-d1d7538d8150', 'e3e3d0bf-2787-48ab-9720-f7cb2a5d4ce2', 'f00d4feb-c74c-4e36-ab50-7989164b3eba', 'c5200c6e-cf74-4b1f-8377-44fbfd9e43ac', '48e0c1c5-8a6b-4724-a313-2b27b5d0073d', '194b348c-e895-4f8d-ab94-c33cb6abd8b7', 'c6537245-d078-41cd-b50c-1df8377c8fd6', '0cec1ac8-30f8-403a-a9b6-97dd88f439b7', 'ca735a43-ae10-4d73-851b-23c32b6e90a2', '0e5e87d8-236e-443c-b231-531694a623f3', '10d09f8d-b8e6-456d-85e8-1ecfa1dd206f', 'ed65677d-a1b5-4cb2-a8c2-fd130551ab54', '85bd5f38-c811-4e68-8e48-2bed178d2a9a', 'e937c7a1-6075-47df-be05-bce50cae3909'} {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[ 1002.658042] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Rules).
[ 1002.708237] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1002.708709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1002.708709] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1002.708935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1002.709028] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1002.709256] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1002.709443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1002.709553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1002.709812] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1002.710126] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1002.710300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1002.710382] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1002.710566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1002.710707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1002.710854] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1002.711232] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1002.711413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1002.711540] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1002.711749] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1002.711886] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1002.712146] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1002.712303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1002.712421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1002.712631] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1002.712770] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1002.713023] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7a1c9ca8afb54adf89f60f26986d891a in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1002.718566] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7a1c9ca8afb54adf89f60f26986d891a
[ 1003.214736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1003.215255] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1003.215255] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1003.363900] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1003.364201] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1003.364201] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1003.364353] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1003.383185] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1003.384328] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Rules Remote IP Prefix).
[ 1003.749355] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1003.749691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1003.749751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1003.749999] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1003.750145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1003.750405] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1003.750560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1003.750678] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1003.750890] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1003.751147] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1003.751304] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1003.751425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1003.751632] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1003.751772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1003.751920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1003.752196] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1003.752391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1003.752604] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1003.752834] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1003.752968] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1003.753208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1003.753366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1003.753488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1003.753725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1003.753868] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1003.754104] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9659f55f47c84e03aaf4a0f5017174c0 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1003.758910] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9659f55f47c84e03aaf4a0f5017174c0
[ 1004.256712] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1004.257280] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1004.257280] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1004.846376] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1004.846839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1004.846955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1004.847319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1004.847546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1004.847892] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1004.848305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1004.848425] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1004.848639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1004.848892] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1004.849054] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1004.849174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1004.849380] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1004.849524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1004.849672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1004.849974] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1004.850202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1004.850371] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1004.850602] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1004.850743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1004.850986] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-463f2928-7993-408c-b4d1-2bd0ea977284 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1004.851142] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1004.851260] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1004.851467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1004.851600] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('463f2928-7993-408c-b4d1-2bd0ea977284',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1004.851880] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 516f075de5234346845944125ab74c60 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1004.859702] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 516f075de5234346845944125ab74c60
[ 1005.075224] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1005.075484] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1005.354685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-463f2928-7993-408c-b4d1-2bd0ea977284" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1005.355182] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1005.355182] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1006.139032] env[60890]: INFO networking_nsxv3.common.synchronization [req-37a0142c-f247-4259-a802-939d01e1e7f7 req-ceb062ef-9ee9-4bf9-87a3-204aa6683596 tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1006.139234] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1006.139552] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1006.139679] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1006.139746] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1006.139945] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1006.140051] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1006.140203] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1006.140302] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1006.140365] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1006.140516] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1006.140653] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1006.140742] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1006.140801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1006.141016] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1006.141081] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1006.141203] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1006.141448] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4e57dc726dfe4ff9a2f4ad539003fe16 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1006.156714] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4e57dc726dfe4ff9a2f4ad539003fe16
[ 1006.378782] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules Remote IP Prefix) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 970 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1006.378988] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1006.378988] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1006.379173] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1006.387808] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules Remote IP Prefix)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1006.393391] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Address Group.
[ 1006.643394] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1006.643547] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1006.643587] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1006.643767] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1006.643858] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1006.644061] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1006.644161] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1006.644221] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1006.644369] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1006.644445] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1006.644552] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1006.644596] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1009.257502] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Address Group taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 121 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 971 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1009.257732] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1009.257732] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1009.257921] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1009.266775] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Address Group" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1009.272457] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Security Group (Rules) Is:1104.
[ 1009.272810] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Security Group (Rules) Are:{'6b38d532-e264-4d27-841f-0e430bb2d26a', '5a93c6de-4566-400e-b225-6fbc784def02', 'aa046b3f-6d44-4acb-8e58-50499e3f8d60', '839be8eb-d3ab-42d2-beae-bb4a6136fea9', '1efe66d8-51ee-4c77-bbc1-b0bff97bd6a0', '83621333-7805-450a-bf42-807dc0903fc0', '02fa8098-695f-4e4c-af31-9d2f20a9fc7c', '2c8665b5-7a16-454b-b09f-403fece4484c', '3c29ea98-dc38-490b-a8e3-abfdc118aa44', 'eb4ebc6c-f4d5-4c08-af40-def126bc447e', '38ad9e85-cb39-4189-9d61-20af96e8fe10', 'a84b8cf6-5370-462a-97e0-5c81e1e22633', 'aeb219c3-95b4-4de5-8fce-a41f8b49395d', '126abd34-fe92-4587-b136-35d78d6b2a21', '80e5b5da-5c33-415d-8683-524bd7b4691a', '2fdf09f9-efa8-4a14-a5c1-0e56ad631495', '01432ff8-955a-47c5-b665-9e3b030f909d', 'f8760856-2ea4-4f89-833e-7c672a2ef54f', 'f37d617f-64a1-4560-85e7-3656703fbd18', '214bfba7-572c-4d07-afe3-1421b94efaff', '26963d6f-35f6-46e6-8a33-94add9111f7b', 'ae07ce08-1e2b-447e-806a-b5f96013bf23', '001325e4-3b30-4168-927e-590103bce85a', 'e00ef03b-af7d-40d2-b5c4-f5bf948cc372', '9c9c140b-f706-4226-bbda-9950cb3e7e0e', '3b8642de-6fce-4721-aa49-9d1d63858bbd', 'f5fdf95e-aaff-43a4-8704-88046c16f0e2', '582e5156-771e-4f49-afec-56511f46df00', 'bedbbbab-f4e5-46f9-9732-8502ac4f5810', '94a8dda8-2e9c-4a69-97ab-ba0bfcc63981', '58c8aa49-9014-43f0-bf5b-d03cbade34c0', '577785e2-d134-40fe-b9a8-9eaf978fb67b', '0a023c7a-aab7-40a3-b867-63f307a7e3f5', '85d4d26a-9348-441b-a876-7f3546795321', '40faa2ad-a6a0-4e51-bc8c-a9fd92e740da', 'dcdd5ccd-96d7-43bb-94e7-a1099f4bfcd4', 'd7206195-0162-4ee9-a6b3-04ceb279bf3a', 'a880ae15-f8b4-4fdc-9f4c-4b0d5cb86453', '77128aab-c69a-4972-bea2-90cce0209793', '8b437b23-1e6f-49fc-a7b2-c532406ff181', '7662670f-bab5-4ef5-9237-479b1e61561d', '44d8f22d-5739-45af-a4ff-362aebd76359', 'd80d761a-2e72-4f5a-bb32-0e68160fbaf8', '3c210669-e296-4518-9563-be9d25f120da', 'ff43b22c-be14-4da5-b80d-36d8c54a860c', 'a30a9427-5e37-4471-9504-44c2d979c5b4', 'e72f919a-2482-420e-9b67-7355fec0316c', 'fcaf713b-1bcf-453c-8091-8e6f34635f0d', 'e014e8c4-efec-481b-9ed2-387c349047ee', 'a50769c7-88fa-4942-9e0e-4cb3595a5c30', 'a3c3f631-daba-43e1-8dec-871740520dc1', '78fd7d31-72e5-4a21-983e-ce48c7a7b713', '238101b5-c86b-4985-a776-3e973c0b60fa', '88520078-a760-425a-ab82-3525ef1c53eb', '92c9eef4-615d-48a3-84a3-cee217f08dce', '8837c3c4-5fbf-426c-a135-efc2f8c355a7', '37199e44-9bde-45c4-98fb-f8c75b0d3a5c', 'ba366f8f-c63c-4119-b420-2c303055cf86', 'f6c2252d-2268-4bb8-bc22-19eddc7610c7', 'aca54de8-c4f7-443e-aaca-7a111e51cc28', '2a81fc84-89bf-45e3-96d5-02a41988759f', '4a52564b-9d2d-469e-ab01-f09f3cdb4bec', '82de9bdb-0c15-45d4-9661-e40f8ec643f5', '13a8ff53-507a-470b-a2f2-fa671d340f4b', 'be36ad87-707f-4732-8a4b-162f66bfff59', '51a8e61b-e2a5-4064-9a56-68018ba339f2', '3b36b955-9c40-4a67-8695-6ebbe0cda689', '306a0b9c-142f-4a95-aa86-5296dbe02cc5', '8a7bf70f-b39d-4901-b42d-15428712deb8', 'a56f9f3d-b945-4159-bef4-429464666018', '6f226f67-b184-4a1c-9718-ac5c0b72e657', '1f67154b-3464-48f3-bfec-9852b358cbdf', 'feee80c5-5ba1-497f-bf8c-7b9bc27cfda0', '9e94d5b9-a9f9-438d-bf22-ab385101ba5b', 'bf9b416e-20fd-48ff-92a5-e238ddb61bd6', '10c57cd4-3944-4147-b429-b5b89c258f97', '670a4800-7ff6-4e55-9b4f-b46b5fb428a3', 'bbe619d9-4b8c-4abd-bc40-7b4017d42fee', '0d9bfa29-bfea-4478-8096-75c79e4dacd7', '0e948d3f-0f2d-4ff5-8578-e597302ee69f', 'e89dc789-cbe5-4c13-a583-6a1301d6acf7', '6e14057c-e840-4c19-9129-61edf2345a07', 'fb8ab3f7-dba9-492a-9091-63d448aff8f2', '0a4dd387-f79f-4f2d-a835-4a5e4fb9b5c2', '0dcb73a7-8912-48d0-a92c-49d62ca12e1d', '839c3392-f7df-4dac-99aa-faaa0ffbc1f2', 'd16d3b0e-751b-4387-9f22-804830b458a3', '1796c9a3-1cf8-4a66-9371-2d52508e5700', 'cbbc9657-8f2d-46bb-862c-6adbd8262def', '3bb43de2-ba3e-4750-9f47-debf358864be', 'ec9b93bf-6a59-4346-9df1-49b056a88ddc', '8b9ec5ff-f9d2-464f-aa26-034bfdedae22', '7011e504-95ac-4051-978d-74e49a6a417b', 'af98123a-50e1-4f30-848d-b652c92345a9', '7b3da12d-92d4-412c-a65e-9a5f565e9883', '14aedcd0-7413-4ff0-921d-0dead6637d4a', '68414d1d-514f-4108-8375-22274b3f2dca', '6cad175e-7e1b-470c-9598-263e629dba58', 'e266007b-24a8-4db4-a5fb-0b1cb7d27945', '18b4bffd-56b4-4aa9-b6a0-f69907fac144', 'ffbd48a8-331c-4efc-bcbc-83d55297b0c5', 'f2e485b8-428f-4e43-8b2c-236f117503d4', '2104385d-94a7-4ae7-8fb9-47feacce37d8', 'e1b59c88-2405-4b3a-9532-73440e2f5282', '37b44564-498b-442c-b259-5368844123d4', 'f2cf7c53-362f-4b7f-b49c-1c4e4bffc040', 'cc605c9b-1ace-499d-b8d8-fdb69ae3ceea', 'e497abb5-a45c-4bc8-b91b-ac72652a730b', '1f1db2e1-ec7e-4773-8ae6-022310685050', '2be9f8fb-fcb3-439a-802b-7c35d9fe987c', '4236e321-4341-469b-b076-101e05a1c634', '0fadcb76-b0b8-4881-b50d-7cc81fe937aa', '01b1ddc1-f902-4110-b9e7-1cb93ba198e3', 'b28c27bc-96c3-4d30-8b23-29cb54f22782', '6f0b1a90-314d-4230-9491-be3fd8160b48', '765a18e9-f395-4ddf-b33a-49588400d911', '0184a453-8434-420b-b0e7-af6dafd6be94', '6a14e5db-843c-4c73-a696-14fe0c37c9a6', '70195330-0f1b-46a5-b7d9-6b6bf9e4072b', '61737029-1c60-45dc-bf45-2b89048815d0', '8fb82bec-30c2-44df-bdf5-44d93a4ca5a5', '1550bb5e-b1a4-4e47-b433-714984a346f8', 'ea5b4a58-a322-479a-8f6f-40f420ac6961', '664e7649-5be7-4789-af43-679bf958c2c4', '0b100977-6974-4db9-ae62-ac6f9bde5f10', 'c6b86557-ee5f-4293-a23d-7962c990d0fb', '2804cc2c-4f0d-48d3-9112-48e6f6f774a4', 'ce0e684d-579e-4e3f-9712-705c0dcd4311', 'dc03a2e0-07d8-4b88-acd1-086899d59d81', 'b7705396-d1cc-4e43-b888-51eb0ddc7e58', '786ef165-7fde-4078-8740-fd9959125b95', '66387184-b7a8-4a37-be9b-a395c1435289', '4d89026d-4a87-43cd-89e4-854240fe7c6c', '41057d22-19a8-4ff7-ac1d-bc20e62be8cb', '4c8da2fb-5b24-4232-be92-e283831004b0', 'bb568b7c-9ed4-4a23-a24d-826aeae5bf41', 'e3654747-6ce5-4c34-9e69-227614e33008', 'a7c899e3-1e84-40d3-9d54-5c9ff71f8c66', '2e3533c7-7ed1-4a27-90f2-244125fcd07b', '2961c826-18bf-491d-91a2-2578410a18e5', '6f41e73c-277c-4be0-b93b-716a4236d8c3', 'e6fd6dd3-caf6-4e6f-8c22-0a6b73c984ff', 'ec10ba18-ca05-4a69-aac3-cb5e549941e4', 'a6f8758f-2516-448e-b4ef-6d330700c92b', 'fffa6845-9346-4dc4-8d2e-6da0881eac27', 'bb162c56-48da-4b1e-ac87-01508b852ed5', 'b6caa5d9-eccd-42f8-874a-8431edde4e92', 'f418017b-6e72-4702-8512-db7d24311693', 'd11614a3-711c-444d-9ddb-2c5adf8bdac3', '6f97d337-3e1c-475f-bd3b-7b17823178be', 'c22400fd-9ed7-4e1a-aa18-f1d92060ea9c', '0b55b0f0-df16-4b41-81d0-2ca9c5bebf24', 'a8261488-7d47-4511-87f1-4b70c625c70e', '55b71a8c-01f1-4e61-8b26-4ad3dae6dd79', 'fb4f905b-4a17-453c-b75e-b4ce53ef2d71', '4a1ff831-4b30-4354-bfb7-c0e0b77bf90c', 'e0d50e7c-efe8-4547-9e43-cb13e88bd4a6', '1947f963-faf1-407c-9995-b29c79427cb3', 'f380c991-18a3-45aa-9613-78b861f34dfe', 'f04d980b-f7d9-4cb3-9d9e-e4e7d419f9b4', '009cb6a7-916a-4da9-a2bf-c8d7a528d2f1', 'a6a45cee-bfda-4847-836d-5b6a15bb6d61', '1456471e-5a27-44c8-97d8-9cdb7d993ef6', '73ec4e34-3f1c-47b3-865e-c49be02a1450', '889f5e8e-b65a-4b4d-b007-72804b983a88', 'dccdd215-7b22-458d-9bc3-45907211a8d0', 'ca7c53f5-cf65-4e22-add6-b32f11f73781', 'd1234b73-ab53-4889-8eec-a220eb91044a', 'cc9e4024-2a52-40ee-b508-c42b42999bdd', 'a04c855f-2aa5-4c62-83d5-439effff24df', '04b45e82-4967-45f4-9139-e8c50f261183', 'd9c01ffd-4146-45cb-9ea3-3e3965976f33', '7be99172-ff42-4033-82c4-835b0a7cee87', 'cb1cefeb-b766-4610-8d3e-19e0c163381b', 'ab12b59c-62e4-4f5f-9fa4-4fab0bf511ee', '4ae77807-7394-4fa4-b332-6eea374322c3', 'a95dc000-d4c1-4bd7-bd1f-ff8845c97940', 'f0b34f41-c6bc-46e7-be6b-3d655bf3c2fa', '9f20533d-1fd8-4673-a050-284b825543ba', '59cd27d4-8ffc-42a1-bb7a-dc2c137b1933', 'ed9d63ef-ed9b-4000-9c9f-d97486bb6313', 'f198c82e-e1b7-4eca-aaf2-7c18868f4600', 'ce40d545-6550-4aeb-a7f8-a735b53b8141', 'af954bb5-a10f-4799-a3d0-10a0d55313bd', '577cd8aa-d141-4ac8-8b74-29ff70a034a0', 'b4e81915-13ab-4309-ada8-9791e009e78c', '9641406d-76e3-44cf-953d-d62bb3cc38ea', '3fad8bbb-41d0-4ba8-8a5b-2b0d23302f5a', '600ee48b-6826-48b5-aadf-b58abd34f5fb', '9291a66a-88d9-4f40-a93e-02b5b5d3a937', '464c0160-2f24-4146-b155-ef58aa98875d', '85306219-2678-44ae-8ee6-700b28ca0c16', '69dfed45-5682-4fa8-bf65-d94bf0a958de', 'eecea5f3-9ecf-44ba-a1fb-bb0e38a8bae5', 'b78ca953-47e4-4994-b97b-d5f30cb26160', '45da1fa0-516a-41f1-beaf-7b0047938b36', 'a0000fb0-481e-407e-9a95-a78fc3f44d25', 'd2c8d550-8a95-4b31-b00e-12fdb6ac3bc7', '4d6fe570-837e-4a57-8d3d-49be88b2101e', '3d623604-6f61-4d33-8501-7ad81f063617', 'c568d725-1788-4353-9dff-531c9bae7d67', '954fa2fd-1746-4b01-ad9e-aefde97c7f30', '01dbf841-f24b-4efb-ad32-65d63130f31b', 'fb5a9539-4230-48e4-9fce-b0cd6e6ab6d0', '63e33c7c-ad25-4466-9ea5-137a5a078ef4', 'fc596c57-8872-47e1-8e7f-e8f866d86176', 'be5b657d-31ee-4a84-9250-d007fe761bef', 'fbc889a9-3e3b-4f64-a82d-4a5e945ae91a', '1cc479c9-58ac-496b-b318-42eca2b4a176', '5b08f0e5-1871-47aa-b176-6c28e431cd64', '4e2e5f41-a617-4156-8637-a2b47a5a2bba', 'd1cacce0-33c1-460a-9c13-6c9bb82d5bb7', '1438c41c-3724-446e-9470-6b6d07775655', 'ee2b287e-1426-49c1-b950-eed64810790c', 'ee9399ac-b225-4d37-9bcb-ae77e14aa39d', '254d4d15-0a7b-4020-bc2f-72258c1c8eec', '1fdc786b-94fd-4481-9215-6709736862a1', 'af1f2fce-4e16-4cb6-8e10-37d3243d2e5c', '971a0755-80ec-4c36-bfd2-3d93101a5abd', '11dd23d4-bbf9-410f-bcbb-2b9b647cf24c', '3a0ee21b-ebf4-43dc-a5fb-920833d4cce1', '5c3fece4-9b5c-4a46-99e8-2a6e7a265c7a', '55860ec2-2462-400b-95bd-99a3222c063b', '1dacba0e-f104-44f4-9b0a-f9bd23c6b6a8', '58293dc1-5c7f-429b-ba74-e9d42d1666f3', 'a00cec11-d330-4a1e-ad66-91971ce5b9a6', 'e9c089e4-563a-427c-b932-4a216df08a57', '94a34b61-b616-4caa-9c26-c3155d20d919', 'd2afd0b0-c524-4918-969f-433a9ebacbd6', '143a19a4-7516-4d81-886e-da40672df71d', '785f1ec8-1ef3-403e-b97e-4a8a678a5939', '8afd7b42-456d-48e4-883e-5c3a5d379be2', '209e8f21-7a8a-4a2b-832f-6b32d1ec0ed9', 'd25ba1a7-6e37-46ed-86ad-4f4bfb7df314', '78b4e008-49cd-470c-ae8a-774ba969b7c3', '49e89fb7-3aba-428d-8f04-160ef9bcf002', 'a22e0506-a966-4a0b-a991-bb4a8408cff2', '30db5f24-5503-4749-b8e2-821dc53792e7', '6674dba3-bb9a-49ae-84c3-a2ddd00e8ad6', 'c0603c93-df1e-40bd-8ce9-6418a127d6b0', '5666f6ff-3d69-4235-994c-452c0fae18fd', 'c948f1fb-4350-4d5d-b1df-fb3a14ac4817', '3a3e9ade-6c09-4828-8407-26f3e688d5e5', 'c906dc90-d7ca-405d-a45b-815c94a71b7e', 'fecb9d13-77f6-4006-a1bc-bf7d840d9f5c', 'fb9757bf-6622-4822-8f32-038495aadc6c', '4896d82f-b449-4189-b263-fd11ab18490d', 'c84fcc4d-1a97-4cb2-8925-4874ee6137e5', '2cac0691-5a91-4653-8f6e-74a883d2af5d', '38111bd3-4034-41e6-9ccc-782c5ae900e7', '1ffbc453-234f-4598-abeb-445d743b2d73', '589f9f17-d562-405d-807a-d06ca5f19245', 'bc7a86f5-a363-4d65-a3d1-5cf5eeb5513c', '08bdd2f9-f2af-40e5-92c1-55503fd797de', '68ff0746-0354-4f85-bd0e-e18806b5264b', '09ebdcdf-8b9b-4517-a588-00e15d641284', 'b965b829-c919-45d7-8f57-bf91f7115ff4', 'e16906cf-be6a-4cd2-a595-03325aa3f6ac', 'd6b8dcbb-e14f-4f11-bb76-f75033b893f8', '9f12fd63-092b-4cf3-8856-28beeef3485e', 'db061e8f-981d-4327-af00-5df74d7c5d30', '5ffed754-cb53-48b6-b90e-7a84d20cbcff', 'e1836ab1-010a-4a7d-b4e4-17caa2e57c69', 'e9d3bb15-9510-4ecb-9944-7b670002f3db', '4aca9e0d-b421-4d5c-97b8-1ff1c05076ac', '8ac9c40e-78a9-4ada-ae66-4da570b5cd9e', '24153d44-a4e5-4967-970c-02cc1409cfc4', 'a2c2c5b1-61d6-480e-987c-36b99a885833', 'faa09b75-06b1-4c1e-923e-770d142e3387', 'fedc0335-bfdb-431c-9ebe-2d0c5dcca610', 'e817d23f-de35-451c-8fd2-ca9c50526fc0', 'a0e22a54-72bd-41e3-9f5a-2fb084d17ddb', '6a3f10a2-a6b1-4d76-934c-dd426e139ba0', '56dbb27f-1ce7-404f-8a52-a85fdcc16099', '42659b80-961d-4ac8-8dd1-c8abdd91e151', 'ffdaaf16-0c6b-4a84-8f94-ec37d83fbf93', '8a00232b-280a-49aa-96cc-4db769877bdd', '770d3355-bdad-40a7-9165-8c87037ccff9', '43da1381-ceee-4b61-9744-379c9888eebd', 'f73e1b31-a070-4800-9257-1b25323936c5', 'c44bf0b3-e403-433b-90da-1f246c6f80f2', 'b1aa9247-ff1b-4b54-8d12-128f806a3983', '69ff7038-e436-4385-9ace-76489f5274a4', '9c9b2b37-25f3-4ea4-9705-88a83120332f', 'fe26cac7-fdcc-4383-a30c-d8de8a4d3c21', '16bf5aeb-22b7-4088-8df3-2ec0f278a90c', '60b683d5-e141-4352-93cc-20bcdd311f74', 'c424aa8b-f99b-43d9-8517-385e52405670', '35a53a62-2092-428a-af42-c36aa4a2e4e1', '1d5b1c86-5db9-4624-8257-59f524e8ce7b', 'ccf33da8-9d54-45f0-b267-2705cf6a4186', '7dc11649-1c7d-4afc-ae36-afbc6a0eb062', 'e0fa8e7f-6095-4051-971f-3d56e8b47ee8', 'a43d51e2-fcce-4c08-944b-cf9ce5083ab7', 'e3450d94-5f06-487c-8f1c-328bc6fba19e', 'b1797cf8-cb1f-4ebb-ac16-6ed10106b9ff', '3af8c6cd-f2d4-4c35-8e15-38a6e8156123', 'dcf5b496-a31a-4753-968e-77837002ca21', 'fff09ab7-441d-4a0f-ae90-674a718ec4f3', 'a4f9530d-abf9-4fd5-b392-1c74596d99d7', 'fa8397df-4f59-44b5-9c06-5d5820e0bd7a', '964275dd-587b-4d13-a15c-5e2e765dc173', '8238829c-bcc9-47ae-8b4a-f56da8d9bc6b', '662725bd-a5e5-456f-b838-563b326e3a69', 'f7cb0d08-2713-4610-ba93-1d22dd5b1e0a', '8f3dd00b-3728-4286-be1f-f6049b5ede30', 'd36e049d-d9b8-4376-97f5-c845ff70dedd', '2c2b4845-5027-4121-b64e-4d71a1abc374', 'cd57e4da-789f-4a64-bbf7-ce1d90673fc9', '5b92549a-ed5c-49bd-b865-19eaa2482b09', 'b6bd256b-05fb-4b80-8c50-7ec02c917143', '346f57b0-657d-4301-97c8-44b7290e6430', 'faf7e41c-e9f5-404b-9221-0c6b012ee740', '3d87c3d1-4f69-4fcb-8b9d-da1f031cdf07', 'cec17d99-408e-4595-9216-0339c1b1a12f', 'c4bcdb1b-9690-4b93-a9b8-55e2e3a3ea9b', '8121b060-9737-4559-b17f-75f227ac9048', 'c4de4049-5c54-449f-94db-e624dc87ed63', '7a5788cc-73ae-44da-b85e-b25ef06234e3', '3a74b557-b525-4dc0-a0e7-e14d50c91d0a', 'cdb1148a-23a6-4b57-b907-03ffc91195a8', '823e793f-a8d3-40fe-be20-df3b5ad9179b', '5aa069f2-bf6c-4c7c-8fec-a00897397e56', '1cf9d498-da4a-4180-94d2-0d5c00afe650', '6aac90cc-9579-460d-b2ed-04a81c109de8', 'e544e2a3-735d-4503-afc2-47d009f7935f', '7b08d4dd-a0b4-43cc-9d2a-972b135c1430', 'c64b8f84-967c-4071-b291-34ba48437ccf', 'fbaf7512-2908-4177-8ed2-42a865d610d6', '62753820-42fb-40da-b36e-a401f377357a', '6a3758ec-79fc-48ca-aac0-e30d12d3b6d9', '3ca70d72-b4f9-41e4-b92c-8c029e1cd2d7', 'a65dbb38-6e75-41df-ba67-0e189e721981', 'ce9c7edd-74b1-4fc3-b808-b611a0c61df5', 'f7b396d3-bdfa-4346-b07f-e6b751236aad', 'f3778041-429e-4499-9e3a-0a0ec8f16936', '165d33c2-263b-401d-91df-0117a29be480', '68fd8c46-2475-4e55-a9b3-42ee20771cdc', '945fb4ab-2407-4ec6-95ab-d9be92246a1e', '6c4fda67-9ddc-4459-afa0-be683844dd0e', '25c8a813-432c-476f-aeaf-b73c7efc2337', '711a3536-173b-45d0-9b9a-4d98244ffc01', 'ce1fd1a6-558d-4261-bc5a-b9b1d3568adf', 'f8132790-8261-4615-82bc-c992a0dc4d0c', '92026d5c-2af4-4b91-9583-fc592f5f71c0', '1bf44a11-d803-436f-87d8-597ce52acd11', '388fd340-e59b-4d1c-9db4-11b4ca6a731b', '609deb9b-c3df-44cf-a011-5b0ae760bf55', '7f558621-b3d8-43f9-b03e-954ad5175cec', '9e35f44e-6f91-4f66-8b91-436d0a57466e', '97e79043-346f-442f-94a1-fe8e792245a1', 'a44385e7-cb47-4f8d-9530-d060cdbb30db', '3dd464bf-f81b-4f06-91f9-9cb3c015b4a3', '1284ecfe-4f71-4dff-a589-8ae7cec46a06', '4649c51c-cca0-4694-973a-c9be70500ecd', 'eac9fec0-2641-4426-8f8b-3a48a0b32821', 'ad07e91e-1c49-48ce-9ae1-335ee19c8982', '1446656e-6f64-47a3-8a54-95b85571d145', 'f1bf0c6b-53a3-4503-811c-e63935f6cedb', '623f3cf6-106a-4a41-b707-92839c6859e1', '17f1a49c-29d8-4874-b0ef-5dc9a69026b1', '73f26647-937f-4b0e-baf7-5f7d891ab022', '6566d6a4-8755-4145-9c53-c66b367852fd', 'acfd2e4a-e422-4863-b861-bd4c1cbfab87', '27eca136-22f1-4f3b-a5f4-47f2e3f5ef28', 'ba02ed24-3ea0-4d76-ad58-055b1ca60908', '1502208c-631a-4cc1-bd71-30d4da929264', 'd72bc675-a968-4981-ae8d-4b8104a8fc94', 'a41600f1-eaf3-467d-bda1-3490acc6c6ee', 'b6829360-ebaa-4356-b51b-2856c130a04b', '95e4fb59-1a70-4dc6-b2e1-3a1b5cd6f0cb', '5e85b3c2-afd2-486a-8324-3da82026e742', '584726a4-ee3c-4cfd-873b-5c09940aace4', '4da78a20-e58e-44ef-90b3-719d970e3608', '5452a771-253f-4a3e-a7d6-3878af982983', '8be25d2e-54f2-45ad-aec4-f0c41d0c6960', '8e6b865b-b83a-4ec0-9137-268a6bbf2d46', 'eed75b04-4032-4491-8700-12abb6192242', 'e18fb7e2-a107-49e1-8559-c8a453a0c12c', 'f1e12046-5ff0-48da-978b-5054e910d3d5', '4e7b5bb8-5dc9-44ba-bd86-01b2fcca4f35', 'e065b96f-c7a7-4109-9a98-0abc08279f41', '6fc1a1f7-48c0-466d-a902-be41bb2909dc', 'c3558061-f684-4797-b74c-55e8de4a383d', '279e5a6f-86ae-47dc-934a-ae06083151a2', '84b96d48-bc75-4aaf-8339-ae9a2bae6798', 'efc90b59-f1dd-4eea-99f9-3b9a58b88483', 'd140e637-4054-4d2f-9545-669a57df1ea8', '3db62012-d8bc-4017-ba31-98fb2b592936', '3e024ad4-7bc8-4446-a722-e846d3e48e0b', 'ede7815e-f2b4-4e92-a8d4-e7f530e83c70', 'f3ef8423-bbec-4546-8d54-c4d2d9eefb3d', 'c1939d81-71d8-402a-a72d-eed70e81ec98', '1af34663-8243-424b-b749-35aaff17c778', '9a21c4b4-f54d-4459-96c3-c9e6ae9e8e31', 'd421d774-ddc0-4979-a5a6-cef76679e898', 'a053d09b-356c-4582-8595-542852913eaf', 'bfc6b91c-e131-47a7-81fa-88e5eb3c5fff', '241e9854-bde0-4e7c-be1c-dea37a2f0e90', '4ca6f1e9-0c4e-474e-a1cb-d41125407f0d', 'd4253539-4006-4dc4-a613-30ebad87735b', '40da80a4-7fc6-4866-a0e7-cc1154336ea6', 'e2c554bb-3bf6-479a-983b-5e58478ab704', 'b8416495-ecf8-4e60-816a-0db10f2c5d2e', '1230ed50-7883-4176-9792-c60e73edc163', '2c269fc8-c778-4161-8be5-10533e66ac9e', 'c647a34a-44bf-41fe-a35f-1481ecc31b22', '0bcc9b1e-9da3-455a-a32e-4c1324c91bd7', '09e4d63a-c555-45b0-8879-05ed1216b496', '77b2202a-fd3e-412c-8aa7-ffc8b0cf8f50', 'fdae3cbf-c8b9-47ad-b362-9e1cdcc23b87', '4af1a18a-6733-468e-8cac-856f966abb42', 'c0996d40-8bae-4a49-965c-513c1adb118c', 'c20809a9-47ae-4379-ac6e-d799c88889da', '45a5677c-96a7-4a37-89d6-9b7035f0ab14', 'fa9bcefb-95e5-4595-89e6-77d7b9e49f82', '9ad76aeb-e965-4d2a-a9de-92f04fb9b8e4', '79f4e8e4-d10b-411c-b47f-7969eec84600', '7597ca1e-0b70-4a71-bb82-922c1a9b52a2', '608073a3-82f6-4577-b38b-71ede8712317', 'f42f230f-d6ce-464e-8025-66a23d253f1a', '02da5604-ce37-4d80-b7c0-dad352952562', '8829a113-df8c-4cc5-8530-5b9d1f318fd8', '6af90a39-eb33-40d8-a4a4-a656b94df580', 'c639d0f5-bba5-4c41-bbc6-c92ce0a01206', '332bb205-cae2-4814-85b5-1225b0d42faa', '757c7c0d-a608-4026-b7b2-8764639b83c7', '9bfa128c-a182-43b5-b5e4-ccccbe0be779', 'fe982f5e-8999-46cc-8ae2-5c0610479a91', 'c908b30c-f7bc-48e7-9b07-5f652939eb79', '289b8cb5-827b-4b96-b41c-146d1d612207', 'f00d8ffa-e5ec-4126-8232-b3e64acef8a1', '7a58b996-fabf-44f8-abb4-3ec0d3dbfeaf', '49e061d2-9378-4f1b-9816-71de3f03f01a', '29571673-755e-4d4c-9728-1abff3f8fd67', '6c0987df-8e1e-411c-b19d-adf095a1050f', 'adb1a1ad-e570-4fdf-9488-538bde8d32f5', 'ab612c9d-05e2-4ab1-a2d4-9146bd0bb6d4', 'c4a645d6-4eb7-4201-99ae-fe3faa482744', '1e051928-63f4-4795-9722-b10db743f215', '37dc3b34-d2d1-4a5a-8b3c-eb401e782bdb', 'bc493fef-5f41-4377-8a24-001b00f280e7', '0725ccf8-beed-4f7f-b741-fc54ff7e2ebd', 'f2f1f0e6-1821-4054-bdef-4f3c76348178', 'e911664c-510f-4c67-8ff6-d67968070e84', '218da72c-3ddd-423e-86f7-d0aed4bd1e47', '5e33dc80-b973-4d0d-ab5a-c6a45198254b', 'ca441aa2-6796-4db8-960b-a7977e434d43', '58a6fe6d-8bef-4419-95cd-9fc604bdc421', '8365d17b-d86d-43e0-852e-f3fda8eae2d8', 'e594aff5-132c-4d9a-952e-1fc74d243a12', '958c0b6a-6f8a-4f6c-8406-f556f75af78a', 'e40a313f-2d65-4606-90be-d168ff24a3f7', '4d21ad8f-4e3b-49a4-96fe-8f66364a0d0c', '1abbf587-b255-4721-b8f0-2b96dc32a646', '1185d64d-2d10-4e81-b958-ee93bab94ae5', '7cd32ef2-1ef1-4f42-84b4-1cdcb23ab30a', '56de7b18-ad03-4d6d-bfb4-aea2c90a6d47', '08f8156c-4637-4906-b6b2-fcaec641b75a', 'a3da1b8c-c49b-466e-b18d-613f8ddc3b05', '34a104f7-6762-4110-ae85-4f192a1db9a5', '92779f0d-dd69-4016-beee-33e8705804a9', '1f9b0305-e137-447b-b185-53a69184efd6', 'b49295ac-1d35-4294-bba8-6aef3b287be6', 'fba18575-c2d9-41e3-83ba-ec8fcf23c984', '79fe113b-9305-40b6-91b2-c9f6abd84439', 'd3ad96e0-1d90-49e8-af43-e3d38e6f7128', 'dbaec471-51bf-4ec1-9822-7a92e9f1b77e', '93dae16f-4011-4ff2-a6d1-c013cf6dbd77', '2c8fc7c8-df5a-4914-b0d7-83b71000b835', '5e83a431-918f-463a-966c-6b402e6021cb', '1be0f48f-0825-45a4-9b3a-913bf013e327', '6838c4a5-f996-4a37-b8a2-15e2a0a4ffcb', '2f8ba902-14bb-4b79-a17f-aeff5c95c84a', '8ecfd421-bd58-45b9-b323-979d6e3630b0', '580aa329-04e3-45e2-8acb-fe14cb26013f', '04e49400-b1fd-43a1-a29b-b2b7deffb80d', '131fa278-c4ed-4af0-b2f1-1fb805dc2a51', 'f8530d7d-3d78-4c4d-a62c-e627e9d28065', '87e1bac4-6dfd-4dd2-ba6c-bf8a90c5453a', '4a26f80f-e99b-40a7-af80-ddd13ab9d582', '6fffeb0e-5192-4855-b76c-325a13629478', 'b65e9824-88c2-4a62-aaef-e317a555dff3', '1244147d-e91f-4a43-9903-ec672032953a', 'a3e59377-ee7a-4002-a95b-02dee37cae6f', 'e261ceb3-7a43-41a8-b870-105307106d6c', 'dbea1d53-4616-40a5-b389-56118fc689c3', '0ecd2cdd-7394-4aae-935f-083440f44c73', 'f830a09d-6403-46af-b959-8c9068eccf11', '85e6e031-95fc-4ce2-9737-b9b457a11759', 'e42b080f-17cf-42de-9fde-5a75f5d6da7f', '28e519e8-50db-407f-afc0-b7bb0f37ecbb', '066a337d-30d8-4291-a38c-3c98191037df', 'f29fc06c-9bea-4105-8982-01646af59e1e', '28dd8e74-e4d3-49fe-9918-84cc4ec0f499', 'ee3929a9-b431-4bc9-9334-fd309a38f152', 'd664efb9-22b5-4dc3-b65c-0b8ac6b52c2e', '3398388f-e461-4c4f-8673-88cd40aee29e', '358a54e2-b53b-44b4-98a4-d301e8b0d87c', '583533dc-a2d6-44dc-a848-e70aad3c5ee8', '11623846-afc1-4b5d-a7a8-70cf65f99d71', 'b174a6c2-747b-41d1-b7fe-88d2ef3b94fb', '2f50427d-088e-47a2-a79f-c9f02938e184', 'f4b13771-8208-4490-9e07-10bb00f4ec2d', '74960b3c-125c-469d-b324-99edb8902762', 'a0ff327a-4777-4ea5-a473-953e99f48fce', '1d6fe346-cbba-4f0e-ad78-ce674e51562e', '21882b2c-8f85-4025-a05e-9d5ac5c7dfe5', '8676b965-8b56-49ab-b6a9-a54292a4c53f', '022cc406-b4f2-420a-b707-82b789638d65', '712394e2-8cde-4fc1-828f-1ea228fb85a6', '3f0f56ed-0942-4f28-b2ab-e015705a388d', '1ff1ff9e-a535-450b-b5e5-2b3aad2a92e1', 'ee0feec7-0fc2-4f57-8eea-f3d9e2999db0', '11782be5-5a4f-4d91-a1f7-66a85f90ce4f', 'd3681b6e-accf-4e7a-8599-9732dff3b868', '9809779f-51b0-45e9-b693-edca5cfa39d2', 'b9b299b3-6398-452e-9651-6189a49e5fea', '010843b8-b1a4-4ece-95e1-d8430a677f49', '7300c61a-eec4-49e5-a60b-1ac573d1c2ca', '3e8314ac-6de4-4fe0-b3b8-8090e7238d54', '53437029-9e9d-4417-8d26-d440ce43ab8c', 'd3a2479e-d6f6-4769-ab79-f8f399872ac2', '3dc7a690-4000-4337-9052-3c175d5b17d1', '984cf644-5297-4ea0-8e48-171a2b9c84c1', 'ed944c6d-d921-486d-96e8-2d616bba357c', 'fca489be-e88c-4540-b01f-3eaa4cdc376e', '777fabc7-034e-42c9-a21b-bd8346081baf', '2e646167-2df2-497e-a3a3-3c3ed4e65936', 'a68a8b48-b54e-4fda-b0fe-f939c87763ad', 'dfc6871c-16b2-426b-8d7e-f339a06594ac', '63047547-b550-47c9-b993-2e886acbf560', '5812319e-e240-4a68-ae1c-eff464ba0a7e', '7e185c0b-5d07-4465-b049-bd18fc2048c9', 'acc316ea-6132-40a7-bdaa-c82630a45736', '9faf5a46-083c-4779-a318-0a209d1bfff9', '1c25d563-9932-41d3-acba-b0ca5aefe8c0', '12afa242-3b02-471b-92ae-01a7a666d975', 'd27cb9f0-260e-457e-811f-ffd1448009cc', 'e638a2b9-395d-46aa-9f48-ad893802cb22', 'e16d1536-3624-47ea-a0b2-4be9049977d5', 'c9ef6859-d2a4-4586-8477-efafa7d2937f', '048381bd-f60c-40c8-85c9-867a04ffe5b5', 'd944bff4-eef4-466d-abc4-a0323e8486f7', '6baa86ac-ebc5-4808-94cb-30af7cca84ac', 'fa71937f-fc79-464b-ac33-3f84913bc5c1', '79cba46b-3a56-43d4-a711-ed1c27f8a8ac', '2ef579a7-8663-40f0-aef4-ba3069611485', '310f4a54-46a9-43e9-b8e2-df926bc1fc13', 'f61b7271-7b94-43b8-b404-a9037f54d120', 'ccd2e48d-222e-499a-8e5f-ab3d417b96c8', '5377676b-7e17-43de-9646-f00a238b562d', 'c824be34-b6bc-4635-aebb-4714244319bf', '3b813add-6d07-48bf-b3f3-300ea3fb0c85', 'd790a0e2-32f6-48b7-9c6e-dee360e54606', 'b6f88864-38f3-4ce4-843d-1807f88ec790', '56f5050a-0fee-4e03-a7fb-334fc44234f6', '186a9c80-281c-4fda-acd9-271fafd649ad', '8eab6bde-c670-45bd-b91e-094355c25bb7', 'c3abbce1-9d00-40a5-9d38-2aff1aed5129', '9bf93fdd-2fad-4c46-a7a3-896d816da7ac', '28d55ac5-c96d-4867-970a-5a69a0bfa928', 'e4d7cf20-4a9b-46e3-a36b-4dd1d1ce97da', '6dc0566e-ead8-43a5-be56-dec682c7c63f', 'f96e429c-b74c-411b-87f9-eba9d50e53d1', '82879c65-a05d-448b-b0f9-6a7083ed4590', 'cdd34ae5-533a-49a0-ae1b-2dcab7087683', '2effa546-1ce6-449c-b3d7-0564561ec4ac', '8fd62893-b488-41fd-96c5-61a7ff5af1bd', '440ee2f9-d226-4f4f-bde1-d14200833b29', '011229d4-5e90-407c-a670-49ffc98f79d1', '5cb60160-7c7d-4b59-9203-2c5ab07842f1', '65a29211-e00c-439d-a26b-bbb119eb9630', '4c8ba24f-6d56-4d7b-91d9-68aae2ae8f74', 'd3181ea2-2e2e-4151-82a0-6dd3bf9aabba', '1bd6158a-c5e7-47ce-b0f5-6ffdb35d17de', '8ad9e821-9357-4562-aaf1-7bfd3b93fb20', 'ead39029-9ec0-40ce-9bad-b260809b8ced', 'f77386bf-9902-4005-93b1-73c0f9f397bb', 'cde94ddc-3f20-4c89-8a3c-9bbb54d26a99', '78758cbd-9265-478d-92d9-b7cccd2f4751', '8687f805-b0a3-4260-8d2e-3c87b697ce0c', '99a5ca37-83e4-4e3a-9187-587f8a2caf20', '6f9ba684-3830-44cc-8ed9-29e1389d71be', '9dd5a52a-3035-4902-a140-1b44b469fb5c', '697e286d-b7ad-4447-9b92-81fcd59c3236', '60d17e33-b6ec-4501-8063-2b3951911a42', '8af0aa51-01ab-4f51-82c2-9d4649f04066', 'c2b5a36b-d306-4db1-a57a-bfc5066833ff', 'ace9ff04-b28a-44ee-9eca-c9822f577fcd', '047eea06-4eb4-41bf-8874-8bea4d8e056b', 'd7227627-de83-4167-b4e9-1360bb1c99a8', '761e0a99-e835-47b3-adde-5cfcc0d6e48b', 'fbcc04fa-12f2-4242-bdea-9c53ad092f13', 'ce4707a7-28e8-41c7-a022-0d4a7b2a02e8', '1f945598-5c9e-4cb4-802a-74d8682d8cd2', 'ee83cf79-3a92-4dc3-8a01-38c8d2817290', '17deaf2b-ca98-4873-bf41-35e0b79bcddd', '022dee22-57bc-48a6-aec2-307aee481dc8', 'a338029c-77c2-48cb-aaf9-d6708dfcc628', '3b9eb7b6-bc65-4476-bde0-75f07d1a479c', 'cb646236-e11c-4a30-a4a6-b40305062d2c', 'b7d15d9c-e206-4355-bc66-600e5727a2db', 'dc274eba-cdfa-493a-8b22-6e418157eca8', '04ce3382-304c-4c10-b79b-a288e40525d6', '9fcbfee9-0f98-447c-9669-0bc475f8554f', '9df3dcd2-7333-4940-abc6-2902bec425b6', '74965ae7-000a-4b65-8ae0-28821722a4c5', 'c0ea3df6-66d6-494c-af93-599cc8005459', 'a099661f-0f96-4b74-baed-ee0ab9870f16', '37f6587f-2a6a-4021-b5c9-e38c3842e331', '83ba575d-c051-48d8-ab84-0e7a4a0397a8', '3da14946-2583-495e-bc5a-eef658d15c95', '6a51f7e6-7131-46a5-8fd2-a4347e699df6', '4b3860e4-68b3-4a9d-bb0f-c79499310593', '5333c492-671a-4354-9d2f-7d73251f1c39', 'ea7a55e5-3749-40aa-84e0-9738398e69ae', '4adec779-4085-43f9-9749-a4f79f5519eb', 'bdf82a38-e0e8-4f61-a3a9-ab71dd33fec3', 'c2fabf99-bf05-4453-8f83-cc8eca721e65', 'c8670329-b9f2-4472-95b3-14552103e125', '4e7ba94f-e9c2-4941-b6ce-383d82c0f65b', '189500e5-532b-48d1-88cc-579b774e8f7f', '07029ca3-e0b5-4d9d-81c7-22db124dba19', '5dc731b6-a1b1-458f-be71-b41cca193acf', '7d9a6280-47ee-4957-8405-1dcc209613bb', 'd97a3c4c-df2d-4051-8c51-b3078efc48d5', '5715f1ac-2a29-4b6c-97d3-8fa5ae66c115', 'f7d16528-a3c8-4c12-a34b-1b98d3e2b218', '4d45ca7a-94ec-4d4e-8f32-20a567f835e1', 'a9d36583-5e55-4934-9f8e-d8d22497ad27', 'ca83cf8c-07f7-4246-9428-991ce4845a38', '2d918405-702e-48ab-bc40-3dc1da4a7a2e', '86a210bd-92f9-414a-a467-2a842cfadaa9', '80eff49b-b490-4df3-a771-4b0bdae7592b', 'cf4ed4b2-dbba-40f2-abe6-19802aadf541', 'eba3010a-a026-4e9e-94a8-ae0f77724cb1', 'a8068f5c-6f96-4bff-b79d-a3dcfeeaa280', '0abdc94f-1499-4384-b38e-c8823b487f21', '928bfa94-b605-4714-b124-5a25c5e42bd8', '7ed3d189-3a99-4816-a1c7-5678c119d8ba', 'cb30f59e-dd28-4a75-94a0-259942493875', 'dff956e7-f33b-4c03-ae57-9538dd622b91', '8bf15a9a-69cf-4b50-b60a-514b36ae96df', '8f174427-d4fb-4f4f-a59f-5e0345bdadd9', 'a17c90cf-7203-4d92-b33e-0f3831ffe006', 'fc182a83-5472-4e2a-a9dc-7e0012e483c5', 'c7afb8e0-c173-4c1f-ae90-a1c65093017b', '41613cc1-c095-4291-869a-df02557c715c', '7e7eb2c0-d1f3-4347-827b-0acfce995056', '84ab8378-05a9-4ffc-af1b-e77135596962', '733e3537-1785-4484-b0cc-33df13195ad2', '59f24c01-a820-46f9-9ec5-26ccc2bb9b99', '22a79425-10fb-432e-a949-01a37c22c495', 'd40ca857-6493-444f-92a6-52947467ad7c', '02d1f251-1551-4f4b-8ace-5a89a897fba3', '97aae706-e442-4648-a616-eb9daf306a4f', '9969d379-361d-4a47-a876-150d83e0b7ba', '802ebbfe-ccd9-458a-93a4-48509d998992', '5eeb1e61-0ea5-4f40-b98a-4169bde6fa30', '124df5fb-bccf-4daa-847f-7576261e09eb', 'c5e60296-9c4e-4eb9-9df2-26a6d27ac31b', '597170dc-c5e5-40e2-9686-a29391e2f08b', 'fe1c3779-aa50-4fd7-9794-23f1b10f447e', '3851d527-deb9-4e67-9ca9-46a76985faeb', '0902ee22-e88c-4f0e-85b9-5e83fe2f0b30', '4efb2bd4-7b01-43ab-8d5f-b80aac7a71a7', '19ae0fce-1d97-4bd9-95fa-aec4f8cebc52', '6d90d74e-28ed-4911-95e8-51ac37233027', '49e33507-c0b0-4bd7-8209-f6ce0dbf15d0', '0dbe9eea-5849-421d-a500-6cd545a2c12f', 'ec2ed287-28c1-48dd-893b-c8cda0a508b4', 'd51bc6f2-1e50-4673-8e60-eae257787176', 'b5cc1352-4268-4513-a03c-bf633544e2dd', '264dbc8a-7fa1-4220-8d5d-c008824f5870', 'abac75c2-3ed7-47af-8df4-2c1597b67964', '5fe05a90-e94c-4c88-a16f-d82799a9bef0', 'fb50169e-785d-4403-9853-adffc6b5171e', 'e70c7461-4be5-437d-b243-abf069ac3ae6', '6f561a81-b15f-43ae-9cc5-eeb0f9a38a71', '1503fb5a-3fe8-4bf3-9eb2-7a903286756d', '0f3d6f25-b150-4f89-8f19-47e05112e968', '7ee44e78-1c9c-45ca-9757-b59f1531492e', '6aeada42-ec65-4bdf-8831-113036a29ffa', '49cebfb6-383f-4c28-9b4e-9a8c875ca489', '3d817822-730f-4917-9401-22a2a406fe8e', '7d054de9-59a2-449a-83f0-aae1c12fd104', '2b1d43d1-9bc8-4fe7-9f1a-04f6c98097ff', 'a44e1a27-2f77-4160-825f-63e89fc2e3a4', '29d3dc08-b4fd-4b56-815b-5f9fa7d351e4', '7fcfac3d-00eb-4bb6-a342-5833b1f3037c', '02a71b1b-d156-46fd-b8e0-919d3e68ab67', 'e2d02080-838f-441f-ae03-5b4e77cde9c1', 'f7c506e4-0ce6-4792-89cc-3ffd47246bec', '69e19bec-8ad6-406b-80cf-51ab5d80b65f', '37ba87ed-b988-42ab-b71f-19a26582f05f', 'eca7eac3-64d5-4db8-aac1-7d2171b5b594', 'e010414f-d677-4efe-99da-b7b66755d4ca', '6bfb0168-5032-40ec-a757-5f9ceeee0c65', 'e2d76a70-0ae1-40b6-a610-899a66e92716', 'ec03565e-b463-4894-a441-f6eaa1e14497', '1fe06423-042e-413e-aefb-31ea7173c770', '7c4d266b-9ceb-4335-943f-3decf874437f', '09c24c3a-991e-4e6c-86dd-00775ddc9db8', '1a1f374c-636c-4ffd-bba9-795be4d7c2f2', '663a0264-041d-4bed-b616-6e20aeb74b39', 'f43951b3-e42d-409a-aa7e-19dff1102007', '9d22157f-08d5-4c08-9fab-28d2e2ae7b57', 'c6ee3756-e8bb-44c9-b0bb-119e9f03b360', '34bf7959-f4bb-401d-b934-7920d274a088', 'a7616d94-b3fc-42e2-98b2-03bd04c97d0f', 'd312af10-76f9-43c3-9118-db3f81bde31d', '48c74bc3-3e15-4ba7-a5ac-786074cab253', '3e87029f-6dd3-4cbf-8d6e-a4f83bc8bc19', '9c49fa14-3e11-4b12-af33-d1398946ffc3', '9985ed29-19b2-4ca2-a114-d4a5db55a349', 'cf2c4e64-dad1-46be-befd-f8dcc859abcd', '8d96ae9f-9dff-4863-abf3-283f80f1027e', 'ee3322a3-875e-4848-9aa9-bfe069f0ae65', '9e62b556-144d-4381-b9f4-c58077f08685', 'e1b38e4c-4612-4d4e-9489-9953af45bea0', 'e191de29-8260-484b-a6a8-4d5ac2d0faca', 'f41d25da-3599-422e-bc46-f1b0d4c38eba', 'bbf86abc-e599-4bae-b673-973dd4c77365', 'e782a415-6fb6-40c7-b28d-d91e58ed44c5', 'b20561db-0067-42ad-8721-9326a247ffcb', '4481d451-0060-4cd0-87ed-b9a00dee1243', '7f811a3f-c3c9-4981-aecb-6bee8144bc66', '141d3859-76fd-4cb7-b64a-8e89bb6105e2', 'ad19bfc9-a815-4cce-a18f-1e50273a4940', '29ba3db3-4c1c-48c2-9785-aa587880a1d7', 'd534f8cf-207f-4b9c-b3e7-63e5396c976b', '91066cd2-7197-4c51-9c8e-3ac6a7f8685f', '55ae8638-cc22-424d-bc8e-304f641175e1', '21ddf401-722a-4898-b34f-e3dbc81ef122', 'aaf28ac3-74a6-4b67-9d27-2fb8b180f9e3', '5768f84d-bc45-479d-825a-5974920907ee', '1d3ba8be-748f-4508-83ad-056e89ecd39f', '70e9491a-e894-439c-b190-469564028fee', 'fe5e2980-4176-4c9a-8e8b-9cd47c9f7e07', '2d528d3a-e365-4f26-8393-65491081b279', 'bfaa69dd-d009-4377-be48-167890cd0179', '1ecf7494-7210-436d-b926-9e873e5b6529', '0ae7bd49-a7c4-4a09-8011-f70d483760f4', '95cf7921-2ddd-4b0a-b393-82942167c148', '8220e6e0-1274-4067-9362-d1843efa59d4', '2664d12b-7727-4ba7-bb20-6d36d52f12f0', '70edd117-06a0-4f49-bf35-29f4c7de3064', 'ccf74d3d-94c4-4ec0-8f39-eed826c566f5', 'ef765d30-9529-409a-927d-f25515bf1710', 'b3b126c8-2a60-4802-bbaf-1b098e51ecce', 'd015e83c-3cad-4f5e-bd30-4dbe071befaf', '8986e901-9e9b-4ed2-b31d-f2176a202a62', 'e0e50c73-fc45-4794-99e6-aa137290a4e1', '7572b39c-6969-487d-be36-ca0a69fdcb8e', 'af434786-c3ee-46f6-8d10-acc5ec0bec29', 'ba575443-b4fb-48f1-a008-c3aea0628271', '5dfa1861-20e7-4866-b924-cdf621dd0a88', '548d42ba-3381-4893-a39f-2237d2752a23', '9c6e8871-295d-461f-911e-2a16aa881389', 'b4b49aea-8d72-4ee3-81d5-398dd7085be2', 'eb27e4a2-d332-4a29-abcc-6ca01d089d06', 'f4f4208f-01e3-4f3a-9135-4574a328dcee', 'fd1cddde-992f-488c-b63e-eafcae157326', '9e561b7e-ec42-4d1d-9695-d93010948fa2', 'b5b4a96c-c1b6-497e-8191-a4b3f0bc39be', '51b22c8f-efd3-4377-8141-3b16d0476e5d', 'b25ae027-ca8b-4a81-bf78-94ec4c0d66ab', '34b97a40-57f8-4326-bcea-73dd6f2309a0', '7a3f1db2-3f97-4d58-b359-59fb7cd9ca49', '48df3a64-b065-4d84-ad45-92911f48d13e', '8b648bb5-7a71-47ce-811c-2049fdc0273e', '0676c759-6a02-4f46-916f-8bd3c99563c3', 'fe74f926-5f68-454e-a893-149be82f3346', '457f3c6f-f4ec-4749-97f9-117dfc06d4db', '5a728d3c-d769-4afd-853d-9d2552d239a5', '86a1e722-f662-43fc-bd17-037bd6123f06', 'ce3871ad-3efb-4785-a306-e49923889613', 'd1f00b2a-1df6-4873-b29e-3aa53818d384', '1322bff5-1e46-471b-a79b-e5da27677247', '1185d178-8e44-47bd-b4dd-a296e2bf9031', '3d4e3921-e785-454f-8ff1-f625205a8808', '23896746-3201-469b-bdd7-a0c2d3f396c7', 'c2e5ea10-1af1-4814-9cd8-ec569c0fb0de', '447eb45f-4d31-4eb9-98a3-cb5c05fbe1e8', '2e4464aa-97ee-4116-8c9d-b9c99123e0a1', 'e874d62e-9f2c-47d0-b5bf-84a9765b48e5', '1bebebdf-5387-4a03-b616-d4766e9e0630', '1b49a31b-1828-4674-9db4-1823b2c9d607', '31dd60d6-8b0e-4abc-81a3-e7bf5aee1cb6', 'f94f9ada-e246-472f-8fea-233c1b8bbd47', '24a7e24e-8902-4872-acc4-aa3665594333', 'fb4837ae-9d8f-4e78-8039-45d62225e224', 'e0add8c2-9f3d-4557-8317-3fd19c913636', 'd1459ed9-6cbf-4387-ad59-972de8282692', '5c026cf8-d0c0-4784-8058-3a035144b3da', '2d157506-234d-4988-8bd7-88e7c35d19b0', '47420f4c-9e29-401d-a29a-1341d1127262', 'd43ee8ea-6cca-4f7f-9bce-a3a8e5e7dab4', '2645464c-d1cb-4d10-9075-17315791f00a', 'bb97eb22-25db-4f75-88a5-abf0eb3c8af6', '35bf7f9c-a62d-41f5-a3ba-99402d1f1eab', 'c4f807a4-41b6-480e-b973-86ff7e73ab21', '22138115-c9b8-4e58-aa35-367f13b91e17', '92de6bd1-222a-4fb5-bb41-120db189a0f7', 'd46b57f1-3a75-4338-8808-65582f8d94c8', '9f584829-b266-48fd-98cf-e86a4235fcbe', '17aab31e-3103-49c0-b8d6-9759bbe79a84', 'c14c1e16-eae8-4d5b-9ddf-b72d64c43125', '1492808e-c3c2-45e4-8518-cab2612e22e7', '66caa0a9-74a8-4cce-89cc-2312d253c6f7', '5a8460ef-028b-44c6-a4bd-d59dcb0796f7', 'ed20d25b-c10d-4f19-a527-ba3986779d48', '415de96b-e6bb-494e-b35e-966c7dd7a87f', '84fe4c94-9dd2-4322-9fbf-1016ee227291', '27fe07e1-d4f3-44bc-a492-2f9d1ddeb0c7', '302ea960-1230-4bf8-99fe-27b1f9067cc2', '894364b3-fb27-4540-a326-1f5d61b78745', '1684acf9-ef02-4e79-a090-bd3e2a427f05', '023631b0-dcd4-45ee-b15c-814f0cf38a3b', 'd6093c90-218a-4dfa-b851-e6683be1b50f', 'c7a0ff84-0f87-41e5-a8da-5257d8f22fbc', 'b6b65099-73c3-4abb-bd1c-842f26e18531', 'ba4e3148-fcf9-42fb-a0b8-8ab4b52ee982', '1a5ed077-8c88-47e4-ae7e-2008eb4c3e22', 'b6071397-8be8-4827-885a-45bbc6d1fb4c', '9feb48b1-9a04-4ba0-bc24-f63ecedfe23c', '3c6cb31b-9f2b-49df-8b06-50468e85fbf8', '31622995-c2ae-494f-b685-3bc348ebd21d', 'c08ae808-8493-4daa-ab35-daa380ee9df8', '80f8c2fa-c4c9-431c-9ae8-23007b27a85c', '34a888c2-3361-4ab5-a980-e3bfa6efc30d', '0904f757-7efb-44d1-a07c-24ff0833a5bf', '8c4347f0-f777-4bda-921c-83dedf80e1de', 'c92ee54a-0b8e-4a1d-abdc-ae1f5fc7826c', '96153093-3cfe-4742-904d-082fda5ae357', '08b7f452-154f-40dd-b211-38c0ab23a43b', '9c891958-ab84-4fde-a1ca-5d78eeaa394f', '1418516c-8077-40a6-9f51-41b9918afc9f', '024d6120-f86f-4e13-8155-9b20e55d99ad', 'c87fb18d-b128-44be-9e86-8ce394233d49', '34b1ff62-2372-473f-b93f-cca96e2eb61a', '56fe2f0a-c17c-4d2a-93ee-b06d2fbd3dff', 'd38d30d2-4da0-4138-97f9-211255fc288b', 'ab228ff2-1100-4100-ab63-88ea99a7e943', '506dd0b4-fee5-4dcc-874e-f0235bfa2828', '26685da3-b9c4-4ed8-a558-2ca25be64085', '7322b9d3-3a26-4da4-a750-af006850174b', '0514918f-5ffa-4e77-b220-8b64830a40a7', 'bf43b8cb-967c-4523-97f3-011be9762178', '7b5d0d66-16ad-4291-8b06-099a356cea4b', 'd5eddf04-7beb-4a20-a045-33d11424d8cc', '90a0a1ba-c8c2-4e00-ab60-42b4d59cc516', '4b414d7a-cbf0-4624-88de-c805138d9469', '6c173c55-d1c9-4285-9b10-e21d678e7999', 'a241e0cd-2994-4d43-93b2-100bf77f7da7', 'c6422c29-1ec6-4b3c-a7dc-d909e4b6c776', 'e17fa03f-00c2-4084-a0f3-87bf51645714', '5809b5c8-d3fb-4995-bbe9-46b35ba7a837', '501088ff-9114-47f3-a7e7-36457bd82fab', 'f936b0ec-fd29-495c-80b9-96023b5fa460', 'd08b5426-3ef3-46b2-ae28-910d7b7937ca', 'f01c43e6-1957-441b-898f-b2c6273ba646', '3b128da7-919a-4a50-a440-f685220695ab', '8818059e-66b1-4ac0-b638-d427ce75bfc4', 'dde81f60-5a15-44f3-b60c-c7b169e554a3', '16a1b6cc-dced-4ab1-b9be-0b664667a67f', '8819cc44-6594-44a9-a97a-6df141cf952d', '040e807f-1b40-4a81-a096-ea383cdec0ed', '56b7a25d-b800-4320-b4f2-aa1668ecf1da', 'd206b9af-4e0c-4a6f-899f-020da44952d7', '53cb9724-e752-4070-bb8e-da16f0be0ba8', '019aa88e-c52c-497c-a823-a89b306889b1', 'f68874ca-c5fd-498e-902c-d872a7d16ee8', '5d2a5b1f-1089-4250-8aaa-f525a55043d6', '04cbcd12-88b1-4650-a755-8ad8499df957', 'dfff59bb-507a-4c4e-b9fa-2cf08b448a44', 'b1995568-c79d-401a-ba73-390016f8a02b', 'a905c316-7ee3-41d2-9b55-6e12c3317b9a', '29a2382d-fddd-49d1-98ee-d0f129b4994f', 'c9b92b8a-b5d3-4d7a-9229-8a450f5ab16a', 'ce5c0cab-9e87-4057-b6df-4045faf73d59', 'af3b6cf3-10d3-4277-8d01-2f4a9d448120', '96dfd8dd-9c48-4fc9-b886-fcbc74b3a6aa', '5ec36386-f181-4891-a12b-6c37896b6bbd', '371b2cc3-9ef5-420f-923b-b6c32920bac0', '80b20778-c6c4-45d4-8710-e55b22534600', '17f91db9-6a80-49d6-be28-b95abf1b8dbe', '4a9b7c39-7855-4c17-80ab-6a3f09c2eff1', 'd835f022-ee2a-43af-999e-cfeccc326ad0', '9cbe8973-fcb1-48b2-944d-35e1183ccb08', 'e4ef434d-8c32-4783-8bfa-4b355dbede8c', '0a44e499-75e2-497c-9343-f1b4262c8ed5', '791c6bdd-fbd5-4b2d-9d9e-7c2c3d0be271', 'b6e46e39-96f2-4857-8b8c-05946f97398f', '68be3455-0d3d-4252-a25d-e98d1f93959c', 'db22c6ea-fbf0-4591-a740-fdb11f104cd6', '5436bb08-3024-4498-8189-60954bb6c170', '0750c8db-f160-4546-bc94-fb7a61cd5001', '6c07253d-cead-49a8-946f-7fe0844137bc', '54855400-8642-44e4-81cc-7075cfea6f37', 'ac26a1e1-5de8-40c3-84b2-3f604cd7e391', 'fae87739-ac61-4ea0-9923-f2bd1b7bcbb5', '2c0509a5-6ec8-4390-ae2d-02ba73bab59e', '2f006589-53f7-411d-8279-359c69fc2b1f', '6f788244-8f9b-46dd-b5e1-b19f3a5e6131', 'c24041ef-b239-40de-b347-df426e0a2073', 'c7e71994-63b8-4fe3-a6ef-f962925aa112', 'c2d5a1a4-760c-4554-9d08-9992f08d35af', 'f78ecdfd-cb09-4f8d-b5e7-6e1d6adf8f88', '1c2563c0-56d0-4848-b2c8-d85691baa0c2', '549c7520-0777-479c-9090-8c83d7ff44fe', '5b423e62-762a-4ab4-b45d-e3081843e829', '4c1cfa51-39f4-4782-8e18-e32bcdaaa296', 'e79a895e-c380-485d-ab80-5072fdd94906', '0cf1c80e-7b02-4407-b59d-9a87f051217b', '77cf495c-34d0-4bca-9533-835c7062484c', '50ba7074-a1f3-4c47-bdca-ebed33745690', '3776d9dd-4495-42e6-96f1-0900b1a651bc', 'd0351fa3-86b9-4e5f-b602-5cc26b2b4e1a', 'ba7a5c70-ef14-4223-940e-d9b65a5c2f44', '0eb15e18-7b3c-4baf-b6ba-49cce461f02d', 'ddea4258-9527-4c8e-aa23-04c9f39d454b', '74088f01-1e96-469c-94e0-f61dc139ab51', '0afd8c89-7ad5-49d8-a85e-39ed10b37b11', '9787f01c-8a0d-4df6-a587-5d30a565f15b', 'fe2952b9-e54c-4fcf-a3f4-0d1b0cdcf532', '128f391d-c159-49cf-a223-c29d146e605c', '2d422992-adc4-4918-8695-1be3ea54fad0', '5443a9e4-61b3-4a58-b622-e0865e818197', '95cb1ba4-475a-4fba-af27-6e299b8693d6', '78a13011-7ede-43d7-9926-d26a847a0367', 'a944dbaa-8f75-4eff-8e48-3600be37c913', '9ba225a5-a1cc-4ebd-bed6-c5b65c881038', 'e0444955-bb69-4d70-9137-acb2a70d80de', 'a1aff3ec-86a5-44d0-b936-91b5f6019078', '2ddf1d36-57da-4007-8272-96c8e3962564', '304386a2-c613-4f16-9327-3f6a2167058b', 'ea28d9ae-a068-4a1d-b763-7020fc76a49a', '9cb0355f-2e29-4d16-876c-0458a685a829', '52be54e4-79c9-469a-807b-0fb6d455b788', '64fc0b3b-d89c-47a4-baae-fc6d5cca946d', '43dfd9d5-6650-4815-8a95-9e55a0457528', 'd736675f-7033-400a-92c7-757bf805b112', '393cafae-c886-463a-8e0e-5138959e1a67', '51883c97-7a92-461f-a1e7-6d0f3ce88999', 'ef917ef5-ddbb-4398-828b-4467007c4b32', '030ac600-cb66-4641-8f22-af782a925ac0', '2a52bd60-903c-4fbe-97ab-4079853853b4', '847d3905-a68a-47cf-9569-f9ff452358dd', 'e631b00c-caa4-40a0-a619-1938e43365ed', '15ee5af7-6e29-4af8-ba31-65da47953658', 'c29771b5-48ee-4d68-9980-793abaf98b19', 'c10f7bd8-9d31-4d23-b0a9-f340ff3aa1d0', 'a49ae307-917e-4aa5-92ed-478f0bccc1d5', '74b6746c-d008-49ce-b752-583ff964da8c', '234a6ada-1f19-4af0-b30b-2715e9d6827c', '22a33e3b-2807-4070-b4c9-548afecb263a', '4d0530d8-d9dc-4311-b635-8903c20f89cb', 'f52103bb-f7e3-4164-a161-ef984538455a', 'bfbd37dc-5a95-4ffe-a366-16c9b9bf557d', '55ec0987-804b-499d-9d94-5fa8c1f94bb7', '204a52cf-b02c-43eb-a468-09c0d3a8bf5e', 'f840c68d-b8d4-411a-add9-79ddb4a5a01d', 'b0535bfe-3132-4a86-92f5-e737403415f1', '8be7bad4-d871-4e0b-b739-c83dba0a6b48', 'ee97aa03-f966-4850-b6fa-245ad3f06746', '52aa95e0-2ad7-461d-a359-3d6f0ba3bf00', '1ff650aa-2505-4f9f-afdc-62affd4a59f3', '491e29c6-4036-4672-9499-7b8d52381b75', '9139c65b-fffd-48de-9b9a-6f3e8ca3b58f', '3c66e192-a190-4d74-b725-52b496ae4e71', 'b397eafa-4896-45c8-88b7-296ab4c72f90', '4cb9b6e3-3826-4e84-8379-1f90b822e929', '90e9e729-d7f6-4641-ae0b-248da1d41c1b', '57f7700e-fa43-4490-afe8-06a649b1edef', 'eee3c5f8-5463-466a-8f2b-962983fda9d6', '6045b6c3-2ae7-499f-99a6-831538329df6', 'c9d6a978-b2f2-47ba-a226-5fba26d893dd', '89b2bca8-bf9e-4af0-9013-e7fdbd1ca74b', '52ee2513-9fcf-44a7-aec9-05cc0408de42', 'b21a3332-ba3c-4abd-ad81-710b1622e68e', '47e84f03-09ae-430d-9abb-d071de9c8f6f', '108d4049-e88b-475e-a248-afeaae2782ef', '5dc7ce26-be22-44be-b7f7-d7195e3d2a88', 'dda8d9e0-ecd1-43bd-aee7-9a9bd16ec428', '4bd1fd10-9ba1-436c-8752-27b7a4560ddb', '4095fe81-5265-4056-bf0d-9dc5f205f721', 'c19bee4f-1b7f-419f-a258-3024a1e24362', '2b1643a6-dfa2-4737-8167-5d2f44345f62', 'c6664015-b8d1-45c3-9e0f-30ed1fcbaace', 'a80b5b94-9818-4a0b-827e-ba7595f24f7d', '7350bb95-48a2-4e12-84e2-5adc4a27aeb1', 'ddc5f6f6-9c3c-4f88-8f6a-0226adcb5919', 'ba30fa04-e616-49f3-9d46-24e1d872b9a4', '3f9ff912-1e91-4605-9249-c40022e202d7', 'd2c30556-8bc6-4dd4-b2fe-c4c3c4ab5885', '6d3d901a-bdc5-47cd-8883-6b4effd4ff7c', 'e3932a7d-dfe7-44ae-bc26-c40afd5f96bf', 'a39488f8-1cda-4fef-92a3-0b397ee5f847', '696c2f1c-a5a2-4245-813e-3c11aa0c33bc', '4045f62d-ccb1-4d3d-bda4-1d85573774ae', 'ca6d2373-81b7-44ac-9383-0a7b44218b4a', '478c3c4a-8272-4ae3-b40c-3768035edbae', '40633562-4688-40d3-96ca-174c7e57eff2', 'ba1e7fed-7979-4496-9b2b-a296c1905013', 'f293b8f8-d4b6-42f2-bdbc-c596e6b8c9d0', '6e34871f-b153-4397-9f96-f967f99a340a', 'a4a9a24c-9bf6-4b4c-b27d-7928040a6be1', 'ab940563-ed69-49c3-9432-9dd0242c38dc', 'b704f1a5-a183-440d-9620-3a5eb025e872', '8760b3e3-dd7e-47c4-84f7-e6ffc9ed2e28', 'b1913a3c-939a-4179-8747-8894039bb9d2', 'b8bddd94-c115-48e8-836e-d2c0dd86b4c1', 'c81493e0-1680-4dc1-9ea9-342b2c6a2c3d', '120da6e4-646a-4e93-8c48-633ecb6b5463', '5b5148bf-3080-497e-bd0f-ee4e0f043ce7', '92fffecd-d047-4272-93f0-e34fe1095654', '1d12cf0b-ce2c-4a35-ba08-b3c207d5e833', 'b23e5e4f-2045-4010-a8cf-b08f78bb8a7f', '3576ff00-12fe-467d-b0b3-70c17e6fa7fb', 'c60b0ae9-3caa-4f4f-9fea-bc87ec18db01', 'fdd72e16-c00d-4f61-9064-fdab83067032', '7830c781-83c6-4a39-ab31-12e93e49971c', 'ef33333f-13b6-49b1-a93a-7e1814c1579c', '1dfe49de-acd2-4751-a309-010e4c80746e', 'd2667211-fce8-46e3-bb93-9fb6f96e2862', '2ec55b7b-20b7-4cf9-b1af-7eebc566c5fd', 'd29ed45d-0ac5-495e-b8f8-0a3f5bca1d68', 'a64c69a2-30d1-43d1-8168-276cdccf9e9a', '20067308-9771-4a1c-98cf-0897c25af809', '43d796a8-4572-4d85-9070-c0e2e2a12e83', 'fb3ef62d-77fc-4235-a38a-1cb6a8f5988b', '7a09516b-1eb1-497e-a6c2-2cbe33bcdaf1', 'a9dca19f-c4e5-423f-b3dd-42b299f03a05', '3775bd14-c4a6-451c-8b65-a80427853797', '100a2224-3dec-4029-b3b2-83910acd126d', '3878a9bc-f6d7-420e-b7cc-1c01a5b88a7d', '1f99ff9c-e148-4911-a841-72fc13a6715c', 'e7627f2d-71ea-47af-9726-506d71cb7008', 'b4a105c3-b28f-4ff4-bbf0-d077055c4047', 'b2064e5c-3e87-4d5b-bf26-f72f6a7f3057', '8bffddb8-9f0d-4702-97f8-260774038c37', 'f99db900-76dd-4e80-ad56-1b5ba62ed0b6', 'e4ceda65-fe1e-4e04-bc5b-095ef9d287ab', 'a06b53a5-5d79-453f-acee-6acc4233380d', 'b3bdb06f-cbba-45ca-b220-9ff15c0a70b3', '9e17b192-16d3-4e67-94d9-f60c52b202b2', 'd7216c28-bc23-4e22-8070-7ca66d10fbaa', '2abb3181-da0b-43ab-84b4-b05f9a71006c', '0783acfc-6542-4e0d-95dc-688beee0abaa', 'adb284a2-bad5-4764-b598-4ab9d292e393', '4e56ac00-e29a-425d-97dd-76a0eca4185c', '731a9eaf-948e-49c0-b104-291038854e3d', '1fc924be-e382-4bd0-b044-98cb6337cacf', 'd1051d18-d9e9-4cfb-9d34-f68b4f116570', '14ddd434-dce1-4c75-a4b6-ab9074d0eab2', '50ec2195-9903-46fa-b266-69167a128a5b', '200bbbcf-9a01-41b5-8b06-954e88759a42', 'ab8310f9-d92e-42e4-83d5-f04b8a826418', '6c9ebcca-bff1-479e-bb26-217cad961da4', '544d33c7-2d7b-4ceb-8cc6-bbfacada9ba9', 'df0a6bfc-a117-4ec8-b2b8-7da7eccd33d8', '31086c8b-ca13-444e-b1aa-e37ecde17fab', '82fccaa9-2976-405b-bc44-444cff9c2815', '6dff6735-c2c5-46b0-8a41-69167bf2fdf7', '5dcdcebb-f642-4dce-a91e-c5a5cc7295bc', '34c6808c-3d1b-4940-bad9-194e18819726', '914b7b3d-7c0f-4f3b-a1d9-38f22996809c', 'b4d9941c-dbf9-4f20-a7fe-bc18cdd88f46', 'ec479475-c53c-4241-8214-59f5cbb0cba0', '2e52f6c4-8141-45a1-96bc-b013a4f4250a', '3ea47f07-37cd-4213-8b55-0987189f3fb5', '98f8dbec-5823-4e7b-b1da-4a0a66dd1841', '6ac1b426-5073-4f20-a6d7-1223585831f1', '3a68d0db-c832-4dfa-9990-900d2ae4052e', 'dc24ea82-98be-49f1-abd8-30a387a82c69', '7b2d625e-8cc2-42fb-9e29-b34aa426c58c', '505644f1-8bcf-443c-910e-56529d51d2ea', '615c020d-21f5-4546-a5ee-fd44d03d85fc', '1d58f4e4-9bea-4bdd-b2c8-cd8f0d9c453b', '63513359-8a62-411b-befc-832b186e26cd', '7d06e6b0-c3f6-4ae1-8ca9-b8caa6855a5d', '85cdf949-7be1-4b5f-92e1-e3727eb8fe94', '387d0ce1-8293-4dce-9768-a1c209033753', 'a0ad096f-db7f-4482-8cc2-7351d09f0e73', 'fce2e2ed-a8d1-4aec-adcf-7994c0652e5b', '3f6b4f1f-7647-4366-a686-c9e305a630c2', '30851f37-ca48-46ec-9eef-a58f074d08ef', '8a0c460a-3834-4fc6-a8d6-e9c8543d36ab', '28b73b0f-d5b1-4281-8525-9fb24f4b217c', '0aea1091-b42a-4c8a-a204-27c189cc5ed8', '6bd99aa1-0b1d-415d-af9c-94f28f1a0042', '919e9c5d-58f7-4390-a195-3a9f9a5171ed', '42600244-505c-4e82-be7b-426daa23b7fd', '9fc4c493-901a-4947-860e-43db9656684b', 'f61d1915-5424-4fb8-9846-ad37acfd3909', '1f2318ea-b550-4abf-b342-a78c067ce3af', '4695169d-6637-40db-b9c3-a14133da2bad', '55f38b46-4ad2-41c6-9723-a684ad69e88d', '456a3a88-691e-4539-9ecf-0f80d149e8c8', '43dec340-cc37-4cf6-8ed3-e1d0f1c61fc4', 'dbcbb0d3-532c-4074-9392-4551fc760b23', 'a04a033f-037a-434a-bd51-e902c27d7cf4'} {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[ 1009.274019] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Segment QoS.
[ 1009.294259] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Segment QoS taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 122 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1009.294395] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1009.294466] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1009.294662] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1009.294749] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Segment QoS" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1009.294860] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Segment QoS Is:0.
[ 1009.294944] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Segment QoS Are:set() {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[ 1009.295016] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] Fetching Policy NSX-T metadata for Type:Security Group (Members).
[ 1011.140615] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1011.141053] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1012.212676] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_service/loopingcall.py, line 150 in _run_loop>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 164 in _sync_all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 125 in all>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 967 in outdated>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 945 in metadata_refresh>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1012.213163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1012.213163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1012.213163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1012.259512] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1012.264979] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] [Policy] The number of outdated resources for Type:Security Group (Members) Is:0.
[ 1012.265123] env[60890]: DEBUG networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Outdated resources of Type:Security Group (Members) Are:set() {{(pid=60890) outdated /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py:1004}}
[ 1012.265261] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Inventory metadata have been refreshed.
[ 1012.265337] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.realization [-] Realizing 20/96 resources of Type:Ports
[ 1012.265418] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 519bac90-d40c-494c-8eb9-6c53e239f10a with Priority: HIGH for action port
[ 1012.265503] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 32332759-ca87-4ef6-955a-325ad700200d with Priority: HIGH for action port
[ 1012.265565] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 03fdd0ff-044d-4309-905b-d51a3bab1b81 with Priority: HIGH for action port
[ 1012.265631] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 55f117f0-ba76-491f-b5c5-7441d3ff7787 with Priority: HIGH for action port
[ 1012.265687] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 9435dc4a-ead0-4afd-a099-668739ae29f4 with Priority: HIGH for action port
[ 1012.265748] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 66d98031-6842-491c-9961-c1b4d0fd5d53 with Priority: HIGH for action port
[ 1012.265807] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 0f875a82-d52b-4617-86ef-09c6a8aeccbb with Priority: HIGH for action port
[ 1012.265862] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 2126120d-06ba-46a5-a74c-77f3b18345ab with Priority: HIGH for action port
[ 1012.265913] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 17b157f7-bf15-4a5c-a271-9af72764322d with Priority: HIGH for action port
[ 1012.265983] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 474946e7-99a3-4290-959e-f2dc1645b999 with Priority: HIGH for action port
[ 1012.266033] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 4fb6c40c-bf99-4b55-9f6b-ca451ded5287 with Priority: HIGH for action port
[ 1012.266086] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 7e152039-3f45-45d1-b2a1-c7acb717f68b with Priority: HIGH for action port
[ 1012.266136] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 1ca72486-d800-43c5-907b-725bb958b3a3 with Priority: HIGH for action port
[ 1012.266189] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 2a1bc88a-7641-4017-874c-f15ee8ec3621 with Priority: HIGH for action port
[ 1012.266237] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: d357f4a7-7fe1-4677-a32d-b2a6c5780592 with Priority: HIGH for action port
[ 1012.266297] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 99c860bb-14da-4e95-b94e-d77d15224b7a with Priority: HIGH for action port
[ 1012.266349] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: fb2c7956-257f-4d1d-8dbb-83b5350798c3 with Priority: HIGH for action port
[ 1012.266406] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: c4585efa-40da-4b2f-81a8-523efec14826 with Priority: HIGH for action port
[ 1012.266460] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: 7e3eabc0-1383-412e-a875-c4e73daf6808 with Priority: HIGH for action port
[ 1012.266512] env[60890]: INFO networking_nsxv3.common.synchronization [-] Enqueued Resource ID: f599de71-cb96-446c-b853-bfcdc671260e with Priority: HIGH for action port
[ 1012.266618] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "all" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1014.528607] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d3372167f2e24714857e745883320afa in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1014.540462] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d3372167f2e24714857e745883320afa
[ 1016.141747] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1016.142513] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=20], Jobs=0
[ 1016.142911] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 519bac90-d40c-494c-8eb9-6c53e239f10a with Priority: HIGH for action port
[ 1016.143285] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 03fdd0ff-044d-4309-905b-d51a3bab1b81 with Priority: HIGH for action port
[ 1016.143617] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 0f875a82-d52b-4617-86ef-09c6a8aeccbb with Priority: HIGH for action port
[ 1016.143869] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: d357f4a7-7fe1-4677-a32d-b2a6c5780592 with Priority: HIGH for action port
[ 1016.144123] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: f599de71-cb96-446c-b853-bfcdc671260e with Priority: HIGH for action port
[ 1016.144364] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7e3eabc0-1383-412e-a875-c4e73daf6808 with Priority: HIGH for action port
[ 1016.144600] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: c4585efa-40da-4b2f-81a8-523efec14826 with Priority: HIGH for action port
[ 1016.144827] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: fb2c7956-257f-4d1d-8dbb-83b5350798c3 with Priority: HIGH for action port
[ 1016.145110] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2a1bc88a-7641-4017-874c-f15ee8ec3621 with Priority: HIGH for action port
[ 1016.145341] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 99c860bb-14da-4e95-b94e-d77d15224b7a with Priority: HIGH for action port
[ 1016.145566] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 1ca72486-d800-43c5-907b-725bb958b3a3 with Priority: HIGH for action port
[ 1016.145783] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 7e152039-3f45-45d1-b2a1-c7acb717f68b with Priority: HIGH for action port
[ 1016.146303] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 4fb6c40c-bf99-4b55-9f6b-ca451ded5287 with Priority: HIGH for action port
[ 1016.146562] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 474946e7-99a3-4290-959e-f2dc1645b999 with Priority: HIGH for action port
[ 1016.146796] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 17b157f7-bf15-4a5c-a271-9af72764322d with Priority: HIGH for action port
[ 1016.147019] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 66d98031-6842-491c-9961-c1b4d0fd5d53 with Priority: HIGH for action port
[ 1016.147231] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 2126120d-06ba-46a5-a74c-77f3b18345ab with Priority: HIGH for action port
[ 1016.147446] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 9435dc4a-ead0-4afd-a099-668739ae29f4 with Priority: HIGH for action port
[ 1016.147674] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 32332759-ca87-4ef6-955a-325ad700200d with Priority: HIGH for action port
[ 1016.147959] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: 55f117f0-ba76-491f-b5c5-7441d3ff7787 with Priority: HIGH for action port
[ 1016.148464] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.148768] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.148959] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.149275] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.149488] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.149759] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-519bac90-d40c-494c-8eb9-6c53e239f10a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.150020] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.150191] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.150463] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.150664] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('519bac90-d40c-494c-8eb9-6c53e239f10a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.150966] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg dc2c8a9617294f25a3557eaea2812677 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.151929] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.152190] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.152348] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.152605] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.152801] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.153051] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-03fdd0ff-044d-4309-905b-d51a3bab1b81 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.153281] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.153431] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.153660] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.153840] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('03fdd0ff-044d-4309-905b-d51a3bab1b81',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.154351] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 78d42f23ba144739b9e0b26f16af4677 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.156195] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.156295] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.156371] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.156533] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.156631] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.156758] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-0f875a82-d52b-4617-86ef-09c6a8aeccbb taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.156845] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.156902] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.157044] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.157117] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0f875a82-d52b-4617-86ef-09c6a8aeccbb',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.157300] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg a8fc6731efde4ecab133cdc0df9737d4 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.157953] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.158018] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.158089] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.158214] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.158288] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.158426] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-d357f4a7-7fe1-4677-a32d-b2a6c5780592 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.158518] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.158580] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.158707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.158773] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('d357f4a7-7fe1-4677-a32d-b2a6c5780592',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.158914] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 262aabd9dbc24775a0b57be6d6a09228 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.159654] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.159761] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.159827] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.160696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.160696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.160696] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-f599de71-cb96-446c-b853-bfcdc671260e taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.160696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.160696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.160696] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.160696] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('f599de71-cb96-446c-b853-bfcdc671260e',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.160696] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ce6e10420f0a43f88e3792bdd88f7d80 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.161240] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.161343] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.161408] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.161551] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.161625] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.161760] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-7e3eabc0-1383-412e-a875-c4e73daf6808 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.161851] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.161911] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.162117] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.162201] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e3eabc0-1383-412e-a875-c4e73daf6808',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.162354] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8bf4b54320784e089aa75715b72f9f92 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.162970] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.163043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.163110] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.163255] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.163329] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.163464] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-c4585efa-40da-4b2f-81a8-523efec14826 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.163556] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.164417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.164417] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.164417] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('c4585efa-40da-4b2f-81a8-523efec14826',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.164417] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2f647c505f7d406fad9ffe4635806b3d in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.164584] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.164707] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.164757] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.164925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.165003] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.165146] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-fb2c7956-257f-4d1d-8dbb-83b5350798c3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.165237] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.165303] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.165455] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.166817] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('fb2c7956-257f-4d1d-8dbb-83b5350798c3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.166817] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4cdd7fea9b1f48cabe1a00c456aa0d51 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.166817] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.166817] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.166817] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.166817] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.166817] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.166817] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2a1bc88a-7641-4017-874c-f15ee8ec3621 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.166817] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.167323] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.167323] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.167323] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2a1bc88a-7641-4017-874c-f15ee8ec3621',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.167323] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 8e932d62d67f4c07a11fd638803f20f5 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.168875] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.168875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.168875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.168875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.168875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.168875] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-99c860bb-14da-4e95-b94e-d77d15224b7a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.168875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.168875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.168875] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.168875] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('99c860bb-14da-4e95-b94e-d77d15224b7a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.168875] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 37ab20360c8c464e985948ba29c6044b in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.169387] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.169387] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.169485] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.169558] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.169633] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.169765] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-1ca72486-d800-43c5-907b-725bb958b3a3 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.169853] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.169909] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.170041] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.170108] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('1ca72486-d800-43c5-907b-725bb958b3a3',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.170236] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 618dfa309fe2453faf4c7dcff3727581 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.170878] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.170984] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.171050] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.171192] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.171267] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.171433] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-7e152039-3f45-45d1-b2a1-c7acb717f68b taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.171526] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.171579] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.171716] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.173147] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('7e152039-3f45-45d1-b2a1-c7acb717f68b',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.173147] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f840aa4480f94d4d8bafa7b46f68fb03 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.173147] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.173147] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.173147] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.173147] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.173147] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.173147] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.173147] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.173568] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.173568] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.173568] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('4fb6c40c-bf99-4b55-9f6b-ca451ded5287',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.173568] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 5beb107d13cd411cac1e0289a8cbc524 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.175097] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.175097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.175097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.175097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.175097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.175097] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-474946e7-99a3-4290-959e-f2dc1645b999 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.175097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.175097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.175097] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.175097] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('474946e7-99a3-4290-959e-f2dc1645b999',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.175097] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 2057cf10489e43c882e4d88662e79495 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.178386] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.178386] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-17b157f7-bf15-4a5c-a271-9af72764322d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.178386] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('17b157f7-bf15-4a5c-a271-9af72764322d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.178386] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg d2c3ccff9c0440a992388727a11b9ace in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.178386] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.178386] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-66d98031-6842-491c-9961-c1b4d0fd5d53 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.178386] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.178386] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('66d98031-6842-491c-9961-c1b4d0fd5d53',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.178386] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 291477a5fe8140208e3581fae8778c2f in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.180802] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.180802] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.180802] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.180802] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.180802] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.180802] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-2126120d-06ba-46a5-a74c-77f3b18345ab taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.180802] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.180802] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.180802] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.180802] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('2126120d-06ba-46a5-a74c-77f3b18345ab',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.180802] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 69cbd8e2b9c24d9986b9659ae4392421 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.180802] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.181600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.181671] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.181819] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.181897] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.182040] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-9435dc4a-ead0-4afd-a099-668739ae29f4 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.182131] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.182184] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.182407] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.182481] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('9435dc4a-ead0-4afd-a099-668739ae29f4',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.182617] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 868ae48398844238988d3758e26c3732 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.183183] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.183309] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.183380] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.183525] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.183603] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.183740] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-32332759-ca87-4ef6-955a-325ad700200d taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.183830] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.183885] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.184117] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.184117] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('32332759-ca87-4ef6-955a-325ad700200d',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.184221] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 192abbf490554a85a25b591ec9ce89bb in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.184811] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 307 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.184891] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.184970] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.185120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.185195] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.185333] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock port-55f117f0-ba76-491f-b5c5-7441d3ff7787 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 310 in port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1016.185420] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1016.185473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1016.186841] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1016.186841] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('55f117f0-ba76-491f-b5c5-7441d3ff7787',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1016.186841] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 81249438fa6c4a759b2aacab23976ed3 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1016.188486] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg dc2c8a9617294f25a3557eaea2812677
[ 1016.188486] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 78d42f23ba144739b9e0b26f16af4677
[ 1016.188694] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg a8fc6731efde4ecab133cdc0df9737d4
[ 1016.190186] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 262aabd9dbc24775a0b57be6d6a09228
[ 1016.190186] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ce6e10420f0a43f88e3792bdd88f7d80
[ 1016.190186] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8bf4b54320784e089aa75715b72f9f92
[ 1016.190186] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2f647c505f7d406fad9ffe4635806b3d
[ 1016.190537] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4cdd7fea9b1f48cabe1a00c456aa0d51
[ 1016.192157] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8e932d62d67f4c07a11fd638803f20f5
[ 1016.192157] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 37ab20360c8c464e985948ba29c6044b
[ 1016.192157] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 618dfa309fe2453faf4c7dcff3727581
[ 1016.192157] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f840aa4480f94d4d8bafa7b46f68fb03
[ 1016.193806] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5beb107d13cd411cac1e0289a8cbc524
[ 1016.193806] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2057cf10489e43c882e4d88662e79495
[ 1016.193806] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-c4585efa-40da-4b2f-81a8-523efec14826" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.193806] env[60890]: Traceback (most recent call last):
[ 1016.193806] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/poll.py", line 111, in wait
[ 1016.193806] env[60890]:     listener.cb(fileno)
[ 1016.193806] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.193806] env[60890]:     result = function(*args, **kwargs)
[ 1016.193806] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.193806] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.193806] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.193806] env[60890]:     return method(*args, **kwargs)
[ 1016.193806] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.193806] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.193806] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.193806] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.193806] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.193806] env[60890]:     result = self.transport._send(
[ 1016.193806] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.193806] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.193806] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.193806] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.193806] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.193806] env[60890]:     raise result
[ 1016.193806] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.193806] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.193806] env[60890]: Removing descriptor: 26
[ 1016.194965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-03fdd0ff-044d-4309-905b-d51a3bab1b81" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.194965] env[60890]: Traceback (most recent call last):
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/poll.py", line 111, in wait
[ 1016.194965] env[60890]:     listener.cb(fileno)
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.194965] env[60890]:     result = function(*args, **kwargs)
[ 1016.194965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.194965] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.194965] env[60890]:     return method(*args, **kwargs)
[ 1016.194965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.194965] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.194965] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.194965] env[60890]:     result = self.transport._send(
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.194965] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.194965] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.194965] env[60890]:     raise result
[ 1016.194965] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.194965] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.194965] env[60890]: Removing descriptor: 42
[ 1016.194965] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-0f875a82-d52b-4617-86ef-09c6a8aeccbb" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.194965] env[60890]: Traceback (most recent call last):
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/poll.py", line 111, in wait
[ 1016.194965] env[60890]:     listener.cb(fileno)
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.194965] env[60890]:     result = function(*args, **kwargs)
[ 1016.194965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.194965] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.194965] env[60890]:     return method(*args, **kwargs)
[ 1016.194965] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.194965] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.194965] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.194965] env[60890]:     result = self.transport._send(
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.194965] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.194965] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.194965] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.194965] env[60890]:     raise result
[ 1016.194965] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.194965] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.194965] env[60890]: Removing descriptor: 32
[ 1016.198791] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-99c860bb-14da-4e95-b94e-d77d15224b7a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.198791] env[60890]: Traceback (most recent call last):
[ 1016.198791] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/poll.py", line 111, in wait
[ 1016.198791] env[60890]:     listener.cb(fileno)
[ 1016.198791] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.198791] env[60890]:     result = function(*args, **kwargs)
[ 1016.198791] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.198791] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.198791] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.198791] env[60890]:     return method(*args, **kwargs)
[ 1016.198791] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.198791] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.198791] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.198791] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.198791] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.198791] env[60890]:     result = self.transport._send(
[ 1016.198791] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.198791] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.198791] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.198791] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.198791] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.198791] env[60890]:     raise result
[ 1016.198791] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.198791] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.198791] env[60890]: Removing descriptor: 24
[ 1016.198791] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d2c3ccff9c0440a992388727a11b9ace
[ 1016.198791] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 291477a5fe8140208e3581fae8778c2f
[ 1016.198791] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 69cbd8e2b9c24d9986b9659ae4392421
[ 1016.201888] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 868ae48398844238988d3758e26c3732
[ 1016.203928] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 192abbf490554a85a25b591ec9ce89bb
[ 1016.205625] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 81249438fa6c4a759b2aacab23976ed3
[ 1016.687517] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-1ca72486-d800-43c5-907b-725bb958b3a3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.687772] env[60890]: Traceback (most recent call last):
[ 1016.687772] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.687772] env[60890]:     timer()
[ 1016.687772] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.687772] env[60890]:     cb(*args, **kw)
[ 1016.687772] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.687772] env[60890]:     result = function(*args, **kwargs)
[ 1016.687772] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.687772] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.687772] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.687772] env[60890]:     return method(*args, **kwargs)
[ 1016.687772] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.687772] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.687772] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.687772] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.687772] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.687772] env[60890]:     result = self.transport._send(
[ 1016.687772] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.687772] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.688577] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.688577] env[60890]:     raise result
[ 1016.688577] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.688577] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.688577] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-474946e7-99a3-4290-959e-f2dc1645b999" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.688577] env[60890]: Traceback (most recent call last):
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.688577] env[60890]:     timer()
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.688577] env[60890]:     cb(*args, **kw)
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.688577] env[60890]:     result = function(*args, **kwargs)
[ 1016.688577] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.688577] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.688577] env[60890]:     return method(*args, **kwargs)
[ 1016.688577] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.688577] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.688577] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.688577] env[60890]:     result = self.transport._send(
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.688577] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.688577] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.688577] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.688577] env[60890]:     raise result
[ 1016.688577] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.688577] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.689900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-7e152039-3f45-45d1-b2a1-c7acb717f68b" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.689900] env[60890]: Traceback (most recent call last):
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.689900] env[60890]:     timer()
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.689900] env[60890]:     cb(*args, **kw)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.689900] env[60890]:     result = function(*args, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.689900] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.689900] env[60890]:     return method(*args, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.689900] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.689900] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.689900] env[60890]:     result = self.transport._send(
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.689900] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.689900] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.689900] env[60890]:     raise result
[ 1016.689900] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.689900] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.689900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-17b157f7-bf15-4a5c-a271-9af72764322d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.689900] env[60890]: Traceback (most recent call last):
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.689900] env[60890]:     timer()
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.689900] env[60890]:     cb(*args, **kw)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.689900] env[60890]:     result = function(*args, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.689900] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.689900] env[60890]:     return method(*args, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.689900] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.689900] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.689900] env[60890]:     result = self.transport._send(
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.689900] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.689900] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.689900] env[60890]:     raise result
[ 1016.689900] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.689900] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.689900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-9435dc4a-ead0-4afd-a099-668739ae29f4" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.689900] env[60890]: Traceback (most recent call last):
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.689900] env[60890]:     timer()
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.689900] env[60890]:     cb(*args, **kw)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.689900] env[60890]:     result = function(*args, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.689900] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.689900] env[60890]:     return method(*args, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.689900] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.689900] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.689900] env[60890]:     result = self.transport._send(
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.689900] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.689900] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.689900] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.689900] env[60890]:     raise result
[ 1016.689900] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.689900] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.689900] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-fb2c7956-257f-4d1d-8dbb-83b5350798c3" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.693206] env[60890]: Traceback (most recent call last):
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.693206] env[60890]:     timer()
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.693206] env[60890]:     cb(*args, **kw)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.693206] env[60890]:     result = function(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.693206] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.693206] env[60890]:     return method(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.693206] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.693206] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.693206] env[60890]:     result = self.transport._send(
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.693206] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.693206] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.693206] env[60890]:     raise result
[ 1016.693206] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.693206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.693206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-d357f4a7-7fe1-4677-a32d-b2a6c5780592" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.693206] env[60890]: Traceback (most recent call last):
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.693206] env[60890]:     timer()
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.693206] env[60890]:     cb(*args, **kw)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.693206] env[60890]:     result = function(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.693206] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.693206] env[60890]:     return method(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.693206] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.693206] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.693206] env[60890]:     result = self.transport._send(
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.693206] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.693206] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.693206] env[60890]:     raise result
[ 1016.693206] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.693206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.693206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-f599de71-cb96-446c-b853-bfcdc671260e" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.693206] env[60890]: Traceback (most recent call last):
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.693206] env[60890]:     timer()
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.693206] env[60890]:     cb(*args, **kw)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.693206] env[60890]:     result = function(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.693206] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.693206] env[60890]:     return method(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.693206] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.693206] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.693206] env[60890]:     result = self.transport._send(
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.693206] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.693206] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.693206] env[60890]:     raise result
[ 1016.693206] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.693206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.693206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2a1bc88a-7641-4017-874c-f15ee8ec3621" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.693206] env[60890]: Traceback (most recent call last):
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.693206] env[60890]:     timer()
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.693206] env[60890]:     cb(*args, **kw)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.693206] env[60890]:     result = function(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.693206] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.693206] env[60890]:     return method(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.693206] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.693206] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.693206] env[60890]:     result = self.transport._send(
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.693206] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.693206] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.693206] env[60890]:     raise result
[ 1016.693206] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.693206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.693206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-7e3eabc0-1383-412e-a875-c4e73daf6808" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.693206] env[60890]: Traceback (most recent call last):
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.693206] env[60890]:     timer()
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.693206] env[60890]:     cb(*args, **kw)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.693206] env[60890]:     result = function(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.693206] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.693206] env[60890]:     return method(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.693206] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.693206] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.693206] env[60890]:     result = self.transport._send(
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.693206] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.693206] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.693206] env[60890]:     raise result
[ 1016.693206] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.693206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.693206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-4fb6c40c-bf99-4b55-9f6b-ca451ded5287" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.693206] env[60890]: Traceback (most recent call last):
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.693206] env[60890]:     timer()
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.693206] env[60890]:     cb(*args, **kw)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.693206] env[60890]:     result = function(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.693206] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.693206] env[60890]:     return method(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.693206] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.693206] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.693206] env[60890]:     result = self.transport._send(
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.693206] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.693206] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.693206] env[60890]:     raise result
[ 1016.693206] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.693206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.693206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-519bac90-d40c-494c-8eb9-6c53e239f10a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.693206] env[60890]: Traceback (most recent call last):
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.693206] env[60890]:     timer()
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.693206] env[60890]:     cb(*args, **kw)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.693206] env[60890]:     result = function(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.693206] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.693206] env[60890]:     return method(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.693206] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.693206] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.693206] env[60890]:     result = self.transport._send(
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.693206] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.693206] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.693206] env[60890]:     raise result
[ 1016.693206] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.693206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.693206] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-2126120d-06ba-46a5-a74c-77f3b18345ab" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.693206] env[60890]: Traceback (most recent call last):
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.693206] env[60890]:     timer()
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.693206] env[60890]:     cb(*args, **kw)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.693206] env[60890]:     result = function(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.693206] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.693206] env[60890]:     return method(*args, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.693206] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.693206] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.693206] env[60890]:     result = self.transport._send(
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.693206] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.693206] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.693206] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.693206] env[60890]:     raise result
[ 1016.693206] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.693206] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.701412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-32332759-ca87-4ef6-955a-325ad700200d" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.701412] env[60890]: Traceback (most recent call last):
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.701412] env[60890]:     timer()
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.701412] env[60890]:     cb(*args, **kw)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.701412] env[60890]:     result = function(*args, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.701412] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.701412] env[60890]:     return method(*args, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.701412] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.701412] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.701412] env[60890]:     result = self.transport._send(
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.701412] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.701412] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.701412] env[60890]:     raise result
[ 1016.701412] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.701412] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.701412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-66d98031-6842-491c-9961-c1b4d0fd5d53" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.701412] env[60890]: Traceback (most recent call last):
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.701412] env[60890]:     timer()
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.701412] env[60890]:     cb(*args, **kw)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.701412] env[60890]:     result = function(*args, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.701412] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.701412] env[60890]:     return method(*args, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.701412] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.701412] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.701412] env[60890]:     result = self.transport._send(
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.701412] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.701412] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.701412] env[60890]:     raise result
[ 1016.701412] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.701412] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1016.701412] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "port-55f117f0-ba76-491f-b5c5-7441d3ff7787" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1016.701412] env[60890]: Traceback (most recent call last):
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
[ 1016.701412] env[60890]:     timer()
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/hubs/timer.py", line 59, in __call__
[ 1016.701412] env[60890]:     cb(*args, **kw)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py", line 265, in main
[ 1016.701412] env[60890]:     result = function(*args, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 311, in port
[ 1016.701412] env[60890]:     port: dict = self.rpc.get_port(os_id)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1016.701412] env[60890]:     return method(*args, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1016.701412] env[60890]:     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1016.701412] env[60890]:     return self._original_context.call(ctxt, method, **kwargs)
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1016.701412] env[60890]:     result = self.transport._send(
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1016.701412] env[60890]:     return self._driver.send(target, ctxt, message,
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1016.701412] env[60890]:     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1016.701412] env[60890]:   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1016.701412] env[60890]:     raise result
[ 1016.701412] env[60890]: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1016.701412] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1019.354080] env[60890]: INFO networking_nsxv3.common.synchronization [req-755f79c8-c37a-4331-90a1-175f48a8ccaa req-5dffe882-f0ff-4839-a19b-ac1c4a2f9c2b tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1019.354565] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1019.354913] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.355024] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.355177] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.355293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.355354] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1019.355506] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.355600] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.355656] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.355798] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.355943] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.356052] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.356116] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.356231] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.356306] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1019.356391] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1019.356567] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4aaadc72d567405e901b781749437495 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1019.369764] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4aaadc72d567405e901b781749437495
[ 1019.607981] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.608212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.608341] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.608606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.608780] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1019.609018] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.609175] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.609300] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.609520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.609768] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.609926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.610131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.610356] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.610498] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1019.610647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1019.610901] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.611056] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.611174] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.611378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.611511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1019.611746] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.611899] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.612023] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.612305] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.612441] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1019.612671] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 387f50463fac40e09cc0c48b5ad0564d in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1019.618464] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 387f50463fac40e09cc0c48b5ad0564d
[ 1019.858429] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.858640] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.858640] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.858774] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.858863] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1019.859063] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1019.859163] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1019.859248] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1019.859377] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1019.859451] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1019.859531] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1019.859606] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1020.115437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1020.115951] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1020.115951] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1020.654882] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1020.655156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1020.655245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1020.655517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1020.655666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1020.655958] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1020.656162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1020.656291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1020.656511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1020.656759] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1020.656920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1020.657042] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1020.657251] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1020.657395] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1020.657544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1020.657794] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1020.657949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1020.658069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1020.658278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1020.658414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1020.658651] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1020.658806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1020.658925] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1020.659145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1020.659298] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1020.659532] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f4031fe09c71493db2ac5d8b127a5af0 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1020.665104] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f4031fe09c71493db2ac5d8b127a5af0
[ 1021.161787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1021.162314] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1021.162314] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1021.705117] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1021.705424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1021.705424] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1021.705689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1021.705840] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1021.706093] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1021.706248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1021.706367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1021.706581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1021.706821] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1021.706974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1021.707092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1021.707301] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1021.707437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1021.707583] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1021.707828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1021.707980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1021.708132] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1021.708314] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1021.708493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1021.708686] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1021.708839] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1021.708957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1021.709161] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1021.709296] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1021.709515] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d033cf7e095c458dac6a0ed7574a5a25 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1021.714154] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d033cf7e095c458dac6a0ed7574a5a25
[ 1022.211365] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1022.211869] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1022.211869] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1022.747977] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1022.748283] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1022.748348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1022.748592] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1022.748740] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1022.748999] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1022.749158] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1022.749278] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1022.749496] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1022.749740] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1022.749896] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1022.750015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1022.750235] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1022.750366] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1022.750512] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1022.750755] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1022.750909] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1022.751027] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1022.751234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1022.751368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1022.751602] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1022.751754] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1022.751870] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1022.752096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1022.752298] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1022.752540] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d00cdb5c9d6a4bb895cbc5d45f17dc8d in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1022.757189] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d00cdb5c9d6a4bb895cbc5d45f17dc8d
[ 1023.254974] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1023.255446] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1023.255446] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1023.792136] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1023.792460] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1023.792517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1023.792772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1023.792926] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1023.793195] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1023.793351] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1023.793468] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1023.793685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1023.793930] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1023.794085] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1023.794203] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1023.794429] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1023.794570] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1023.794716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1023.795025] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1023.795183] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1023.795302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1023.795509] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1023.795646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1023.795912] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1023.796087] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1023.796210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1023.796417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1023.796553] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1023.796799] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg ed44f4d3161741d2a280bfba60e979ed in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1023.801934] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ed44f4d3161741d2a280bfba60e979ed
[ 1024.299350] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1024.300034] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1024.300034] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1024.355445] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1024.355555] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1024.836030] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1024.836348] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1024.836403] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1024.836643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1024.836790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1024.837067] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1024.837238] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1024.837357] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1024.837577] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1024.837827] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1024.837984] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1024.838107] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1024.838315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1024.838458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1024.838607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1024.838855] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1024.839012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1024.839130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1024.839421] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1024.839572] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1024.839814] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1024.839980] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1024.840141] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1024.840355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1024.840490] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1024.840713] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg d5750c25850b4f8a9f571344d9dabaaa in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1024.845491] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg d5750c25850b4f8a9f571344d9dabaaa
[ 1025.343178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1025.343694] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1025.343694] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1025.879347] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1025.879751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1025.879751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1025.879983] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1025.880152] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1025.880412] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1025.880571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1025.880690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1025.880935] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1025.881185] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1025.881340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1025.881458] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1025.881664] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1025.881806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1025.881954] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1025.882208] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1025.882388] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1025.882526] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1025.882736] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1025.882879] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1025.883138] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1025.883295] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1025.883415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1025.883620] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1025.883759] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1025.884039] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 55264bf6064b425baed6d03e7b7505f1 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1025.888907] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 55264bf6064b425baed6d03e7b7505f1
[ 1026.385894] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1026.386393] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1026.386393] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1026.944065] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1026.944419] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1026.944517] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1026.944769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1026.944977] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1026.945240] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1026.945398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1026.945519] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1026.945735] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1026.945988] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1026.946179] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1026.946306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1026.946515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1026.946657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1026.946806] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1026.947057] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1026.947218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1026.947361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1026.947607] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1026.947781] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1026.948036] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1026.948200] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1026.948322] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1026.948529] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1026.948665] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1026.948893] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e933221064a143118f9794cb227fd520 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1026.954484] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e933221064a143118f9794cb227fd520
[ 1027.450986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1027.451628] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1027.451628] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1028.040782] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1028.041112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1028.041112] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1028.041384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1028.041530] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1028.041791] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1028.041948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1028.042073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1028.042291] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1028.042535] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1028.042689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1028.042818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1028.043029] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1028.043172] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1028.043318] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1028.043640] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1028.043820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1028.043941] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1028.044173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1028.044308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1028.044547] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1028.044701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1028.044818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1028.045065] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1028.045203] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1028.045432] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 4d5700125434468390fc0df74cf58357 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1028.050801] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4d5700125434468390fc0df74cf58357
[ 1028.547701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1028.548236] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1028.548236] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1029.116429] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1029.116726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1029.116814] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1029.117048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1029.117220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1029.117483] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1029.117643] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1029.117763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1029.117979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1029.118223] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1029.118378] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1029.118497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1029.118704] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1029.118844] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1029.118994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1029.119242] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1029.119467] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1029.119609] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1029.119821] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1029.119960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1029.120218] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-0c085ed3-c0af-4caa-9421-049e323e32dc taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1029.120372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1029.120488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1029.120694] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1029.120826] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('0c085ed3-c0af-4caa-9421-049e323e32dc',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1029.121056] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b42af3f76bf54b99be90b310c225124e in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1029.127220] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b42af3f76bf54b99be90b310c225124e
[ 1029.356943] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1029.357118] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1029.623566] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-0c085ed3-c0af-4caa-9421-049e323e32dc" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1029.624020] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1029.624020] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1030.412529] env[60890]: INFO networking_nsxv3.common.synchronization [req-755f79c8-c37a-4331-90a1-175f48a8ccaa req-205d8e98-eb10-4bb2-b469-e115b37b21af tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1030.413352] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1030.413672] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1030.413837] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1030.413895] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1030.414085] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1030.414168] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1030.414324] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1030.414413] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1030.414473] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1030.414665] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1030.414809] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1030.414928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1030.414978] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1030.415113] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1030.415189] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1030.415301] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1030.415490] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg ec979b69b03647828281a828c038bdcc in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1030.422896] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg ec979b69b03647828281a828c038bdcc
[ 1030.918396] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1030.918562] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1030.918609] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1030.918925] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1030.919058] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1030.919287] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1030.919424] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1030.919511] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1030.919700] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1030.919807] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1030.919919] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1030.920049] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1035.414336] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1035.414674] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1040.415647] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1040.415972] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1044.023102] env[60890]: INFO networking_nsxv3.common.synchronization [req-35e61bbb-1c3d-49fe-a5a1-2fcb24b18290 req-f657af39-093a-46e6-9e98-f4b568c33439 tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1044.023636] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1044.023955] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.024098] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.024162] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.024357] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.024443] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.024595] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.024690] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.024749] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.025019] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.025084] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.025131] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.025210] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.025336] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.025414] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.025501] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1044.025677] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg f7afdcb0223f4e2e9854f603bf4e953c in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1044.033977] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f7afdcb0223f4e2e9854f603bf4e953c
[ 1044.274029] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.274220] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.274344] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.274683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.274755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.275034] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.275191] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.275308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.275524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.275775] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.275930] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.276064] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.276273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.276414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.276565] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.276817] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.277013] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.277086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.277298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.277434] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.277667] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.277825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.277944] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.278210] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.278346] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1044.278572] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 850604b977f54e3fadc486bc8c207ad3 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1044.284276] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 850604b977f54e3fadc486bc8c207ad3
[ 1044.528087] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.528087] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.528311] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.528345] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.528482] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.528616] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1044.528717] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1044.528795] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1044.528926] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1044.529014] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.529097] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1044.529172] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.530791] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg df932fb5082b4e6594711672856a3e92 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1044.542357] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg df932fb5082b4e6594711672856a3e92
[ 1044.781271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1044.781617] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1044.781617] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1045.318495] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1045.318772] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1045.318841] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1045.319100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1045.319252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1045.319510] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1045.319668] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1045.319789] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1045.320067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1045.320277] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1045.320433] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1045.320553] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1045.320759] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1045.320900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1045.321050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1045.321299] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1045.321456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1045.321578] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1045.321783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1045.321920] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1045.322157] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1045.322312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1045.322432] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1045.322636] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1045.322769] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1045.323022] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f448ebd62d9f4055a28e73395d1ee7e2 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1045.328061] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f448ebd62d9f4055a28e73395d1ee7e2
[ 1045.824775] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1045.825349] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1045.825349] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1046.361964] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1046.362249] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1046.362319] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1046.362569] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1046.362716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1046.362972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1046.363129] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1046.363250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1046.363465] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1046.363711] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1046.363865] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1046.363986] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1046.364224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1046.364367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1046.364515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1046.364836] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1046.365035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1046.365162] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1046.365375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1046.365514] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1046.365883] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1046.366150] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1046.366355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1046.366685] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1046.366857] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1046.367094] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 5091195611244458902ecaa6af4e474e in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1046.372056] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 5091195611244458902ecaa6af4e474e
[ 1046.869280] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1046.869912] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1046.869912] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1047.405790] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1047.406118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1047.406118] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1047.406374] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1047.406525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1047.406820] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1047.406987] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1047.407110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1047.407364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1047.407613] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1047.407783] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1047.407970] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1047.408187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1047.408333] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1047.408482] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1047.408734] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1047.408892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1047.409012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1047.409223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1047.409362] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1047.409601] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1047.409756] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1047.409875] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1047.410177] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1047.410314] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1047.410544] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 37f0a38459c7433282ac5bc870b23b96 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1047.415584] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 37f0a38459c7433282ac5bc870b23b96
[ 1047.912672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1047.913192] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1047.913192] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1048.449830] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1048.450108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1048.450197] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1048.450464] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1048.450614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1048.450886] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1048.451048] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1048.451167] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1048.451765] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1048.452043] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1048.452218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1048.452343] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1048.452561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1048.452707] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1048.452860] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1048.453117] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1048.453273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1048.453391] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1048.453600] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1048.453734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1048.453974] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1048.454131] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1048.454250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1048.454461] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1048.454598] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1048.454934] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 1054f117883c4524ab52de211cc2f14f in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1048.460424] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 1054f117883c4524ab52de211cc2f14f
[ 1048.957307] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1048.957821] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1048.957821] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1049.025915] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1049.026184] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1049.494971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1049.495377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1049.495524] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1049.495794] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1049.495971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1049.496266] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1049.496428] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1049.496550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1049.496770] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1049.497019] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1049.497178] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1049.497298] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1049.497506] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1049.497649] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1049.497804] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1049.498057] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1049.498218] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1049.498338] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1049.498545] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1049.498681] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1049.498927] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1049.499084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1049.499221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1049.499511] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1049.499655] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1049.499892] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 3b0defc4ab7848f09487fab80c38e45f in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1049.504764] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 3b0defc4ab7848f09487fab80c38e45f
[ 1050.001994] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1050.002521] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1050.002521] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1050.537647] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1050.537998] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1050.538154] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1050.538423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1050.538576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1050.538843] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1050.539003] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1050.539123] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1050.539337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1050.539626] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1050.539784] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1050.539904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1050.540130] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1050.540286] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1050.540431] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1050.540680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1050.540846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1050.540990] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1050.541202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1050.541339] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1050.541575] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1050.541729] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1050.541847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1050.542053] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1050.542189] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1050.542426] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b2da4fe0eefb450497e1b462f6307db4 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1050.547257] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b2da4fe0eefb450497e1b462f6307db4
[ 1051.044674] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1051.045230] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1051.045230] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1051.581305] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1051.581612] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1051.581667] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1051.581906] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1051.582058] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1051.582384] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1051.582562] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1051.582684] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1051.582919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1051.583168] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1051.583316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1051.583436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1051.583645] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1051.583785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1051.583934] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1051.584226] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1051.584394] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1051.584515] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1051.584726] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1051.584898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1051.585142] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1051.585299] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1051.585420] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1051.585628] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1051.585843] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1051.586091] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 240e9ea221864654b4ef6d4a593d1d4a in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1051.591302] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 240e9ea221864654b4ef6d4a593d1d4a
[ 1052.088544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1052.089138] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1052.089138] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1052.651354] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1052.651747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1052.651747] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1052.652050] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1052.652212] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1052.652482] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1052.652639] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1052.652758] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1052.653070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1052.653327] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1052.653487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1052.653606] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1052.653823] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1052.653966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1052.654115] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1052.654366] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1052.654520] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1052.654657] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1052.654904] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1052.655052] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1052.655295] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1052.655448] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1052.655564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1052.655771] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1052.655983] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1052.656241] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 48fb3a9c82184b20af261b3bd72b4222 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1052.661207] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 48fb3a9c82184b20af261b3bd72b4222
[ 1053.157938] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1053.158432] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1053.158432] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1053.696585] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1053.696888] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1053.696962] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1053.697259] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1053.697422] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1053.697685] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1053.697846] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1053.697969] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1053.698194] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1053.698440] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1053.698596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1053.698715] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1053.698939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1053.699083] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1053.699231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1053.699481] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1053.699719] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1053.699845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1053.700122] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1053.700269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1053.700513] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-33d4d7f2-e763-4569-a8bb-6c616b793986 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1053.700670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1053.700790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1053.701006] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1053.701148] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('33d4d7f2-e763-4569-a8bb-6c616b793986',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1053.701379] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 09507e2f32884d1b9ad9018d8a4e8b35 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1053.706253] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 09507e2f32884d1b9ad9018d8a4e8b35
[ 1054.026619] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1054.026840] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1054.203672] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-33d4d7f2-e763-4569-a8bb-6c616b793986" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1054.204213] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1054.204213] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1054.989074] env[60890]: INFO networking_nsxv3.common.synchronization [req-35e61bbb-1c3d-49fe-a5a1-2fcb24b18290 req-0cb5e3cd-d459-4e39-8ad8-66893f6470c2 tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1054.989661] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1054.989993] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1054.990120] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1054.990209] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1054.990389] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1054.990468] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1054.990619] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1054.990711] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1054.990769] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1054.990912] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1054.991042] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1054.991125] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1054.991180] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1054.991307] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1054.991378] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1054.991460] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1054.991639] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 341096bf45d542c7bd83f369276ff8e1 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1054.999170] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 341096bf45d542c7bd83f369276ff8e1
[ 1055.493887] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1055.494130] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1055.494130] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1055.494314] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1055.494409] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1055.494607] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1055.494714] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1055.494825] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1055.494958] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1055.495043] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1055.495128] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1055.495212] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1059.991342] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1059.991700] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1064.992556] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1064.992556] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1068.306578] env[60890]: INFO networking_nsxv3.common.synchronization [req-19cd2099-3f57-4ca9-acf5-8947a3bc0602 req-b55a70c9-abab-454f-afb0-b79ab6a91209 tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1068.306578] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1068.306578] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.306578] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.306578] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.306578] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.306578] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1068.306578] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4d77c500458f440f8c9bc20b07922c3f in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1068.322526] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4d77c500458f440f8c9bc20b07922c3f
[ 1068.562852] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.562852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.562852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.562852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.563093] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.563240] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.563372] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.563481] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.563701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.563971] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.564173] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.564309] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.564550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.564697] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.564847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.565131] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.565292] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.565445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.565670] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.565807] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.566046] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.566202] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.566355] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.566576] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.566719] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1068.566986] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 8494605fd8024595a725c6eda2739340 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1068.571759] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 8494605fd8024595a725c6eda2739340
[ 1068.810653] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.810884] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.810884] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.811133] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.811233] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.811437] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1068.811543] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1068.811606] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1068.811760] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1068.811839] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1068.811925] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1068.812026] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1069.069488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1069.069958] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1069.069958] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1069.608031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1069.608031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1069.608031] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1069.608031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1069.608752] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1069.608852] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1069.609073] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1069.609232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1069.609487] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1069.609663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1069.609943] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1069.610153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1069.610308] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1069.610558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1069.610731] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1069.610993] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg b7171f9ceffb4d4b9bc045c5029b5dd3 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1069.615969] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg b7171f9ceffb4d4b9bc045c5029b5dd3
[ 1070.112918] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1070.113472] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1070.113472] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1070.650079] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1070.650363] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1070.650411] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1070.650689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1070.650838] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1070.651089] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1070.651250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1070.651396] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1070.651618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1070.651863] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1070.652031] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1070.652156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1070.652364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1070.652504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1070.652652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1070.652933] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1070.653096] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1070.653215] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1070.653445] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1070.653559] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1070.653794] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1070.653951] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1070.654071] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1070.654276] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1070.654411] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1070.654633] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 39e74ccedc054913a67f071110a94a26 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1070.659455] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 39e74ccedc054913a67f071110a94a26
[ 1071.156328] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1071.156808] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1071.156808] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1071.697680] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1071.698015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1071.698015] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1071.698269] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1071.698415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1071.698673] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1071.698832] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1071.698952] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1071.699166] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1071.699410] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1071.699564] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1071.699683] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1071.699887] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1071.700041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1071.700192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1071.700439] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1071.700593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1071.700710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1071.700914] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1071.701051] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1071.701284] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1071.701436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1071.701554] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1071.701755] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1071.701893] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1071.702117] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 404d81c7d5084fa19d78362c5632ece8 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1071.706946] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 404d81c7d5084fa19d78362c5632ece8
[ 1072.204232] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1072.204740] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1072.204740] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1072.741074] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1072.741406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1072.741406] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1072.741671] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1072.741818] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1072.742081] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1072.742241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1072.742361] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1072.742584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1072.742835] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1072.742989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1072.743108] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1072.743315] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1072.743454] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1072.743603] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1072.743851] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1072.744017] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1072.744145] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1072.744353] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1072.744488] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1072.744725] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1072.744900] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1072.745021] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1072.745234] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1072.745374] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1072.745599] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg bb64cefe125d47ee83501dcbf7599d20 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1072.750545] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg bb64cefe125d47ee83501dcbf7599d20
[ 1073.248312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1073.248832] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1073.248832] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1073.300165] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1073.300335] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1073.784725] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1073.785105] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1073.785500] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1073.785805] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1073.786025] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1073.786325] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1073.786493] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1073.786616] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1073.786845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1073.787094] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1073.787252] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1073.787373] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1073.787581] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1073.787727] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1073.787876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1073.788157] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1073.788317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1073.788439] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1073.788652] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1073.788790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1073.789030] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1073.789187] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1073.789306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1073.789513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1073.789650] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1073.789872] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 9ae9dfc4c3d4494bb7918b2fc5a15b3b in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1073.794701] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 9ae9dfc4c3d4494bb7918b2fc5a15b3b
[ 1074.292513] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1074.293027] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1074.293027] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1074.533104] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 122d5e289dbf4d159f605300396dca48 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1074.544572] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 122d5e289dbf4d159f605300396dca48
[ 1074.830560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1074.830853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1074.830892] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1074.831156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1074.831306] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1074.831560] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1074.831716] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1074.831851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1074.832095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1074.832347] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1074.832505] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1074.832623] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1074.832843] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1074.833012] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1074.833169] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1074.833422] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1074.833580] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1074.833701] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1074.833936] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1074.834069] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1074.834306] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1074.834463] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1074.834582] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1074.834787] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1074.834976] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1074.835210] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 641c97a5ad87405ca409ab3230708bfc in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1074.839846] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 641c97a5ad87405ca409ab3230708bfc
[ 1075.337044] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1075.337541] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1075.337541] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1075.876027] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1075.876364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1075.876364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1075.876634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1075.876785] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1075.877050] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1075.877206] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1075.877325] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1075.877543] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1075.877791] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1075.877948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1075.878067] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1075.878273] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1075.878413] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1075.878560] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1075.878810] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1075.878966] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1075.879086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1075.879294] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1075.879430] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1075.879670] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1075.879825] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1075.879948] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1075.880236] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1075.880379] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1075.880605] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg e3bba68a7b174574b4feeafd19efdb9d in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1075.885582] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg e3bba68a7b174574b4feeafd19efdb9d
[ 1076.382633] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1076.383143] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1076.383143] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1076.921187] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1076.921502] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1076.921558] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1076.921820] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1076.921988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1076.922250] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1076.922407] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1076.922527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1076.922751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1076.923075] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1076.923265] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1076.923389] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1076.923610] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1076.923751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1076.923902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1076.924210] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1076.924414] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1076.924546] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1076.924763] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1076.924919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1076.925169] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1076.925326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1076.925443] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1076.925659] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1076.925795] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1076.926029] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f8454f25b8984965abe0bda1140ab530 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1076.930976] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f8454f25b8984965abe0bda1140ab530
[ 1077.428723] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1077.429176] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1077.429176] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1077.968109] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1077.968456] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1077.968516] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1077.968746] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1077.968905] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1077.969167] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1077.969327] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1077.969446] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1077.969666] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1077.969914] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1077.970072] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1077.970190] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1077.970398] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1077.970542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1077.970691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1077.970942] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1077.971114] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1077.971245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1077.971453] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1077.971587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1077.971828] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-16e4b6a0-7846-4601-8c4f-b28c943bb567 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1077.971991] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1077.972148] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1077.972360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1077.972496] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('16e4b6a0-7846-4601-8c4f-b28c943bb567',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1077.972725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 25f1db7a1c7d440da1cc254e18122080 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1077.977632] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 25f1db7a1c7d440da1cc254e18122080
[ 1078.301424] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1078.301655] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1078.475618] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-16e4b6a0-7846-4601-8c4f-b28c943bb567" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1078.476169] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1078.476169] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1079.216554] env[60890]: INFO networking_nsxv3.common.synchronization [req-19cd2099-3f57-4ca9-acf5-8947a3bc0602 req-630bc012-d202-4480-b628-8aab70d09fdd tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1079.217457] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1079.217792] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1079.217928] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1079.217997] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1079.218194] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1079.218281] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1079.218464] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1079.218528] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1079.218589] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1079.218739] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1079.218877] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1079.218970] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1079.219022] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1079.219161] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1079.219239] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1079.219323] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1079.219498] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 72d37c9340584b7a807c27d8574e0590 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1079.227162] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 72d37c9340584b7a807c27d8574e0590
[ 1079.721425] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1079.721586] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1079.721657] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1079.721893] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1079.721988] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1079.722183] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1079.722283] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1079.722348] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1079.722499] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1079.722577] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1079.722661] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1079.722742] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1084.218643] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1084.219071] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1089.219363] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1089.219757] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1092.874293] env[60890]: INFO networking_nsxv3.common.synchronization [req-e1b9a560-051a-4f98-bc2c-cdca746ba428 req-fb91c790-3430-4591-8b21-85aefe9d2e15 tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1092.874293] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1092.874293] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1092.874293] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1092.874293] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1092.874293] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1092.875774] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1092.875774] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 27038f0de6ad4ac29f55361b3acd6eaa in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1092.884225] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 27038f0de6ad4ac29f55361b3acd6eaa
[ 1093.120063] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1093.120288] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1093.120417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1093.120691] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1093.120876] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1093.121207] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1093.121377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1093.121497] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1093.121724] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1093.121977] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1093.122134] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1093.122271] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1093.122542] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1093.122907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1093.122907] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1093.123095] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1093.123250] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1093.123367] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1093.123571] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1093.123717] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1093.124066] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1093.124244] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1093.124364] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1093.124625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1093.124763] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1093.125021] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg caec11f07a04463babbed9a549710288 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1093.130382] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg caec11f07a04463babbed9a549710288
[ 1093.377298] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1093.377485] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1093.377485] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1093.377692] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1093.377786] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1093.377977] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1093.378078] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1093.378138] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1093.378291] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1093.378369] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1093.378463] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1093.378542] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1093.629109] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1093.629109] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1093.629109] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1094.165285] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1094.165561] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1094.165599] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1094.165927] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1094.166086] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1094.166345] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1094.166507] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1094.166627] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1094.166847] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1094.167659] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1094.167831] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1094.167955] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1094.168192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1094.168337] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1094.168523] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1094.168789] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1094.168949] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1094.169070] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1094.169281] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1094.169417] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1094.169658] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1094.169845] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1094.170005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1094.170245] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1094.170382] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1094.170614] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg f4f83fa71e4e49b5ba6d12c1d00eeab7 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1094.175198] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg f4f83fa71e4e49b5ba6d12c1d00eeab7
[ 1094.673195] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1094.673697] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1094.673697] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1095.209781] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1095.210092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1095.210149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1095.210384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1095.210534] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1095.210797] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1095.210957] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1095.211113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1095.211340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1095.211586] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1095.211743] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1095.211859] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1095.212084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1095.212230] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1095.212377] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1095.212628] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1095.212786] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1095.212921] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1095.213153] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1095.213293] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1095.213534] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1095.213690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1095.213809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1095.214019] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1095.214178] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1095.214406] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 43c14439b6434602aa5c973cd471f41f in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1095.219552] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 43c14439b6434602aa5c973cd471f41f
[ 1095.716423] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1095.716919] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1095.716919] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1096.252920] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1096.253241] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1096.253302] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1096.253544] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1096.253689] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1096.253951] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1096.254110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1096.254229] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1096.254447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1096.254696] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1096.254853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1096.255011] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1096.255231] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1096.255375] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1096.255527] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1096.255780] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1096.255939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1096.256095] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1096.256312] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1096.256447] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1096.256688] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1096.256842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1096.256960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1096.257171] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1096.257305] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1096.257533] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg cfe41e72402b4ad886d999aaffd375e6 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1096.262296] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg cfe41e72402b4ad886d999aaffd375e6
[ 1096.759953] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1096.760522] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1096.760522] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1097.301657] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1097.301960] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1097.302016] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1097.302267] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1097.302415] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1097.302677] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1097.302848] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1097.302971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1097.303192] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1097.303437] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1097.303593] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1097.303710] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1097.303919] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1097.304074] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1097.304223] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1097.304471] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1097.304625] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1097.304742] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1097.304989] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1097.305137] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1097.305376] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1097.305531] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1097.305647] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1097.305855] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1097.306010] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1097.306250] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2995d509ed8142f5bef1d68c6f69e3da in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1097.311725] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2995d509ed8142f5bef1d68c6f69e3da
[ 1097.808634] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1097.809188] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1097.809188] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1097.873967] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1097.874105] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1098.345705] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1098.345992] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1098.346036] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1098.346289] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1098.346436] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1098.346691] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1098.346851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1098.346975] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1098.347189] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1098.347457] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1098.347614] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1098.347734] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1098.347939] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1098.348100] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1098.348248] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1098.348496] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1098.348654] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1098.348773] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1098.348979] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1098.349113] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1098.349350] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1098.349504] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1098.349621] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1098.349824] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1098.349956] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1098.350186] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 7e5d15b7db094ac284d07d3c18717af5 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1098.355063] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 7e5d15b7db094ac284d07d3c18717af5
[ 1098.852809] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1098.853364] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1098.853364] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1099.416512] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1099.416852] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1099.417102] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1099.417437] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1099.417596] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1099.417869] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1099.418035] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1099.418159] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1099.418384] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1099.418669] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1099.418856] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1099.418981] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1099.419198] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1099.419340] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1099.419490] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1099.419741] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1099.419898] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1099.420079] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1099.420311] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1099.420450] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1099.420693] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1099.420851] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1099.420971] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1099.421180] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1099.421332] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1099.421539] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 2cd8be1c3ef44e5e96e728240d553c52 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1099.426447] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 2cd8be1c3ef44e5e96e728240d553c52
[ 1099.923579] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1099.924128] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1099.924128] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1100.461972] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1100.462303] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1100.462584] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1100.462644] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1100.462769] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1100.463038] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1100.463204] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1100.463326] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1100.463550] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1100.463789] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1100.463946] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1100.464084] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1100.464354] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1100.464510] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1100.464663] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1100.464953] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1100.465110] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1100.465240] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1100.465449] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1100.465587] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1100.465827] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1100.465985] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1100.466106] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1100.466316] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1100.466451] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1100.466681] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 69bfe45d4ba8458185019355772a23a1 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1100.471951] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 69bfe45d4ba8458185019355772a23a1
[ 1100.969205] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1100.969722] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1100.969722] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1101.506459] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1101.506790] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1101.506853] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1101.507149] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1101.507317] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1101.507592] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1101.507751] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1101.507874] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1101.508111] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1101.508365] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1101.508525] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1101.508646] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1101.508863] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1101.509005] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1101.509156] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1101.509411] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1101.509568] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1101.509690] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1101.509902] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1101.510041] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1101.510293] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1101.510483] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1101.510549] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1101.510757] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1101.510894] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1101.511130] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg c8fb1f20ef5947a4a14dce2492645985 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1101.516505] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg c8fb1f20ef5947a4a14dce2492645985
[ 1102.013842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1102.014379] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1102.014379] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1102.550694] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 355 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1102.551030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1102.551030] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1102.551360] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1102.551532] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1102.551797] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock network-231 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 358 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1102.551963] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1102.552098] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1102.552359] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1102.552572] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock Segment taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 64 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 361 in network>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 867 in network_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1102.552725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1102.552842] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1102.553077] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1102.553221] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "Segment" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1102.553368] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "network-231" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1102.553615] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 266 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1102.553768] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1102.553885] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1102.554092] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1102.554224] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1102.554457] env[60890]: DEBUG networking_nsxv3.common.locking [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Lock port-5e550d2c-cbcf-4d4f-b862-96ee222ec515 taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenpool.py, line 87 in _spawn_n_impl>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_green.py, line 69 in __call__>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/futurist/_utils.py, line 45 in run>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py, line 165 in _process_incoming>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 309 in dispatch>, <FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py, line 229 in _do_dispatch>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py, line 79 in get_network_bridge>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 269 in precreate_port>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1102.554608] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquiring lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1102.554725] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1102.554988] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Acquired external semaphore "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1102.555128] env[60890]: DEBUG networking_nsxv3.api.rpc [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method get_port called with arguments ('5e550d2c-cbcf-4d4f-b862-96ee222ec515',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1102.555342] env[60890]: INFO oslo_messaging._drivers.amqpdriver [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Expecting reply to msg 52efe307d74845898633ded5f2d13f01 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1102.561461] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 52efe307d74845898633ded5f2d13f01
[ 1102.874600] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1102.874793] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1103.057563] env[60890]: DEBUG oslo_concurrency.lockutils [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Releasing lock "port-5e550d2c-cbcf-4d4f-b862-96ee222ec515" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server [None req-8e80f78c-1a33-45de-8dfe-57f107f6b717 None None] Exception during message handling: oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1103.058068] env[60890]: ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/agent.py", line 79, in get_network_bridge
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     self.realizer.precreate_port(current["id"], network_meta)
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py", line 270, in precreate_port
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     port: dict = self.rpc.get_port(os_id)
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     return method(*args, **kwargs)
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 174, in get_port
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     return cctxt.call(self.context, 'get_port', host=self.host, port_id=port_id)
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/neutron_lib/rpc.py", line 149, in call
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     return self._original_context.call(ctxt, method, **kwargs)
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/client.py", line 190, in call
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     result = self.transport._send(
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/transport.py", line 123, in _send
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     return self._driver.send(target, ctxt, message,
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 788, in send
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     return self._send(target, ctxt, message, wait_for_reply, timeout,
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server   File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/_drivers/amqpdriver.py", line 780, in _send
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server     raise result
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server oslo_messaging.rpc.client.RemoteError: Remote error: ArgumentError ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server ['Traceback (most recent call last):\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming\n    res = self.dispatcher.dispatch(message)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch\n    return self._do_dispatch(endpoint, method, ctxt, args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch\n    result = func(ctxt, **new_args)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py", line 67, in wrapper\n    return method(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/api/rpc.py", line 311, in get_port\n    port = db.get_port(context, host, port_id)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/oslo_db/sqlalchemy/enginefacade.py", line 998, in wrapper\n    return fn(*args, **kwargs)\n', '  File "/opt/stack/networking-nsx-t/networking_nsxv3/db/db.py", line 179, in get_port\n    ).join(\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/base.py", line 279, in _generative\n    x = fn(self, *args, **kw)\n', '  File "<string>", line 2, in join\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/base.py", line 304, in generate\n    fn(self, *args, **kw)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/orm/query.py", line 2429, in join\n    onclause_element = coercions.expect(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 396, in expect\n    resolved = impl._literal_coercion(\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 899, in _literal_coercion\n    self._raise_for_expected(element)\n', '  File "/opt/stack/data/venv/lib/python3.10/site-packages/sqlalchemy/sql/coercions.py", line 518, in _raise_for_expected\n    raise exc.ArgumentError(msg, code=code) from err\n', "sqlalchemy.exc.ArgumentError: ON clause, typically a SQL expression or ORM relationship attribute expected, got <class 'neutron.plugins.ml2.models.PortBinding'>.\n"].
[ 1103.058068] env[60890]: ERROR oslo_messaging.rpc.server 
[ 1103.861115] env[60890]: INFO networking_nsxv3.common.synchronization [req-e1b9a560-051a-4f98-bc2c-cdca746ba428 req-171375b7-c926-4108-a594-d7a72a16583e tempest-ServerDiskConfigTestJSON-1163112681 tempest-ServerDiskConfigTestJSON-1163112681-project-member] Enqueued Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1103.861115] env[60890]: INFO networking_nsxv3.common.synchronization [-] Processing Resource ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a with Priority: HIGHEST for action security_group_members
[ 1103.861115] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock migration-tracking taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 203 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider.py, line 442 in is_migration_in_progress>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "migration-tracking" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1103.861115] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 206 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1103.861115] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 208 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1103.861115] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1103.861115] env[60890]: DEBUG networking_nsxv3.api.rpc [-] networking_nsxv3.api.rpc.NSXv3ServerRpcApi method has_security_group_used_by_host called with arguments ('ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a',) {} {{(pid=60890) wrapper /opt/stack/data/venv/lib/python3.10/site-packages/oslo_log/helpers.py:65}}
[ 1103.861115] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 4b77b450fae843f89a7a80537257421d in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1103.867792] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 4b77b450fae843f89a7a80537257421d
[ 1104.362281] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Rules) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 902 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 914 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1104.362463] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1104.362526] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1104.362720] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1104.362833] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Rules)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1104.363025] env[60890]: DEBUG networking_nsxv3.common.locking [-] Lock Security Group (Members) taken with stack trace [<FrameSummary file /opt/stack/data/venv/lib/python3.10/site-packages/eventlet/greenthread.py, line 265 in main>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/realization.py, line 222 in security_group_members>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 909 in sg_members_realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 35 in wrapper>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 627 in _realize>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/plugins/ml2/drivers/nsxv3/agent/provider_nsx_policy.py, line 921 in metadata>, <FrameSummary file /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py, line 47 in get_lock>] {{(pid=60890) get_lock /opt/stack/networking-nsx-t/networking_nsxv3/common/locking.py:46}}
[ 1104.363126] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquiring lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:310}}
[ 1104.363185] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:313}}
[ 1104.363333] env[60890]: DEBUG oslo_concurrency.lockutils [-] Acquired external semaphore "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:321}}
[ 1104.363411] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "Security Group (Members)" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1104.363496] env[60890]: INFO networking_nsxv3.plugins.ml2.drivers.nsxv3.agent.provider_nsx_policy [-] Resource: Security Group (Members) with ID: ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a already deleted.
[ 1104.363575] env[60890]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "member-ffac2a6d-4465-4aeb-a844-2d2eca8c3e4a" {{(pid=60890) lock /opt/stack/data/venv/lib/python3.10/site-packages/oslo_concurrency/lockutils.py:331}}
[ 1104.536203] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 6be16d9f76474427a56162f89fcc47bb in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1104.547721] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 6be16d9f76474427a56162f89fcc47bb
[ 1108.858016] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1108.858445] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1113.859428] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1113.859726] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1118.860534] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1118.860534] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1123.861776] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1123.862496] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1128.863926] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1128.864276] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1133.864867] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1133.865267] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0
[ 1134.539900] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Expecting reply to msg 798cdf4d18c94ad89d2f4266ec6d5500 in queue reply_156db6c4256a4c58884afce15e6f5340
[ 1134.559133] env[60890]: INFO oslo_messaging._drivers.amqpdriver [-] Received RPC response for msg 798cdf4d18c94ad89d2f4266ec6d5500
[ 1138.866258] env[60890]: INFO networking_nsxv3.common.synchronization [-] No activity for the last 5 seconds.
[ 1138.866634] env[60890]: INFO networking_nsxv3.common.synchronization [-] Sizes Queue[Active=0, Passive=0], Jobs=0